[HDFS-16187] SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.3.2, 3.2.4
• Component/s: snapshots
• Labels: pull-request-available
• Assignee: Shashikant Banerjee
• Reporter: Srinivasu Majeti
• Created: 26/Aug/21 15:11
• Updated: 12/Jul/22 10:53
• Resolved: 13/Sep/21 04:56

Description

The below test shows the snapshot diff between across snapshots is not consistent with Xattr(EZ here settinh the Xattr) across NN restarts with checkpointed FsImage.

@Test
public void testEncryptionZonesWithSnapshots() throws Exception {
  final Path snapshottable = new Path("/zones");
  fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
      true);
  dfsAdmin.allowSnapshot(snapshottable);
  dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
  fs.createSnapshot(snapshottable, "snap1");
  SnapshotDiffReport report =
      fs.getSnapshotDiffReport(snapshottable, "snap1", "");
  Assert.assertEquals(0, report.getDiffList().size());
  report =
      fs.getSnapshotDiffReport(snapshottable, "snap1", "");
  System.out.println(report);
  Assert.assertEquals(0, report.getDiffList().size());
  fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
  fs.saveNamespace();
  fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
  cluster.restartNameNode(true);
  report =
      fs.getSnapshotDiffReport(snapshottable, "snap1", "");
  Assert.assertEquals(0, report.getDiffList().size());
}

Pre Restart:
Difference between snapshot snap1 and current directory under directory /zones:

Post Restart:
Difference between snapshot snap1 and current directory under directory /zones:
M .

The side effect of this behavior is : distcp with snapshot diff would fail with below error complaining that target cluster has some data changed .

WARN tools.DistCp: The target has been modified since snapshot xxxxx

Issue Links

• is related to: HDFS-16055 Quota is not preserved in snapshot INode
• links to: GitHub Pull Request #3340
• links to: GitHub Pull Request #3640