Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-5633

Improve OfflineImageViewer to use less memory

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Target Version/s:

      Description

      Currently after we rename a file/dir which is included in a snapshot, the file/dir can be linked with two different reference INodes. To avoid saving/loading the inode multiple times in/from FSImage, we use a temporary map to record whether we have visited this inode before.

      However, in OfflineImageViewer (specifically, in ImageLoaderCurrent), the current implementation simply records all the directory inodes. This can take a lot of memory when the fsimage is big. We should only record an inode in the temp map when it is referenced by an INodeReference, just like what we do in FSImageFormat.

        Activity

          People

          • Assignee:
            Jing Zhao
            Reporter:
            Jing Zhao
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development