Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5633

Improve OfflineImageViewer to use less memory

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently after we rename a file/dir which is included in a snapshot, the file/dir can be linked with two different reference INodes. To avoid saving/loading the inode multiple times in/from FSImage, we use a temporary map to record whether we have visited this inode before.

      However, in OfflineImageViewer (specifically, in ImageLoaderCurrent), the current implementation simply records all the directory inodes. This can take a lot of memory when the fsimage is big. We should only record an inode in the temp map when it is referenced by an INodeReference, just like what we do in FSImageFormat.

        Attachments

        1. HDFS-5633.000.patch
          4 kB
          Jing Zhao

          Activity

            People

            • Assignee:
              jingzhao Jing Zhao
              Reporter:
              jingzhao Jing Zhao
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: