[HDFS-13143] SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Component/s: snapshots
• Hadoop Flags: Reviewed
• Assignee: Shashikant Banerjee
• Reporter: Shashikant Banerjee
• Created: 13/Feb/18 17:39
• Updated: 02/Oct/18 22:22
• Resolved: 27/Feb/18 23:40

Description

HDFS-12594 introduced an iterative approach for computing snapshotDiffs over multiple rpc calls. The iterative approach depends upon the startPath (path of the directory with respect to the snapshottable root) and the size of the createdList (0 in case the startPath refers a file) to exactly determine form where in each iteration the calculation has to start.

 

In case of the diff computation between a snapshot and current tree(if any of the snapshot names specified in the getSnapshotDiffReport call is null or empty), the last SnapshotDiff associated with directory/file might change owing to changes in the current tree in between the rpc calls in the absence of a global fsn lock. This might result in consistencies in the snapshotDiffReport.

In case the snapshotDiffReport computation needs to be done between the current tree and a snapshot, we should fall back to non-iterative approach to compute snapshotDiff.

Issue Links

• is related to: HDFS-12594 snapshotDiff fails if the report exceeds the RPC response limit