Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5433

When reloading fsimage during checkpointing, we should clear existing snapshottable directories

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • snapshots
    • None

    Description

      The complete set of snapshottable directories are referenced both via the file system tree and in the SnapshotManager class. It's possible that when the 2NN performs a checkpoint, it will reload its in-memory state based on a new fsimage from the NN, but will not clear the set of snapshottable directories referenced by the SnapshotManager. In this case, the 2NN will write out an fsimage that cannot be loaded, since the integer written to the fsimage indicating the number of snapshottable directories will be out of sync with the actual number of snapshottable directories serialized to the fsimage.

      This is basically the same as HDFS-3835, but for snapshottable directories instead of delegation tokens.

      Attachments

        1. HDFS-5433.patch
          9 kB
          Aaron Myers
        2. HDFS-5433-2.patch
          9 kB
          Andrew Wang

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            atm Aaron Myers
            atm Aaron Myers
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment