Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13143

SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0
    • snapshots
    • None
    • Reviewed

    Description

      HDFS-12594 introduced an iterative approach for computing snapshotDiffs over multiple rpc calls. The iterative approach depends upon the startPath (path of the directory with respect to the snapshottable root) and the size of the createdList (0 in case the startPath refers a file) to exactly determine form where in each iteration the calculation has to start.

       

      In case of the diff computation between a snapshot and current tree(if any of the snapshot names specified in the getSnapshotDiffReport call is null or empty), the last SnapshotDiff associated with directory/file might change owing to changes in the current tree in between the rpc calls in the absence of a global fsn lock. This might result in consistencies in the snapshotDiffReport.

      In case the snapshotDiffReport computation needs to be done between the current tree and a snapshot, we should fall back to non-iterative approach to compute snapshotDiff.

      Attachments

        1. HDFS-13143.001.patch
          4 kB
          Shashikant Banerjee
        2. HDFS-13143.002.patch
          4 kB
          Shashikant Banerjee
        3. HDFS-13143.003_committed.patch
          6 kB
          Tsz-wo Sze
        4. HDFS-13143.003.patch
          6 kB
          Shashikant Banerjee

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shashikant Shashikant Banerjee
            shashikant Shashikant Banerjee
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment