[HDFS-16496] Snapshot diff on snapshotable directory fails with not snapshottable error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: namanode
• Labels: pull-request-available
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 06/Mar/22 21:51
• Updated: 08/Mar/22 11:08
• Resolved: 08/Mar/22 11:08

Description

Running a snapshot diff against some snapshotable folders gives an error:

org.apache.hadoop.hdfs.protocol.SnapshotException: Directory is neither snapshottable nor under a snap root!
	at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.checkAndGetSnapshottableAncestorDir(SnapshotManager.java:395)
	at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:744)
	at org.apache.hadoop.hdfs.server.namenode.FSDirSnapshotOp.getSnapshotDiffReportListing(FSDirSnapshotOp.java:200)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReportListing(FSNamesystem.java:6983)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReportListing(NameNodeRpcServer.java:1977)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getSnapshotDiffReportListing(ClientNamenodeProtocolServerSideTranslatorPB.java:1387)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)

This is caused by HDFS-15483 (in order snapshot delete), and the issue is in the following method in SnapshotManager:

  public INodeDirectory getSnapshottableAncestorDir(final INodesInPath iip)
      throws IOException {
    final String path = iip.getPath();
    final INode inode = iip.getLastINode();
    final INodeDirectory dir;
    if (inode instanceof INodeDirectory) { // THIS SHOULD BE TRUE - change to inode.isDirectory()
      dir = INodeDirectory.valueOf(inode, path);
    } else {
      dir = INodeDirectory.valueOf(iip.getINode(-2), iip.getParentPath());
    }
    if (dir.isSnapshottable()) {
      return dir;
    }
    for (INodeDirectory snapRoot : this.snapshottables.values()) {
      if (dir.isAncestorDirectory(snapRoot)) {
        return snapRoot;
      }
    }
    return null;
  }

After adding some debug, I found the directory which is the snapshot root is not an instance of INodeDirectory, but instead is an "INodeReference$DstReference". I think the directory becomes an instance of this class, if the directory is renamed and one of its children has been moved out of another snapshot.

The fix is simple - just check `inode.isDirectory()` instead.

Issue Links

• is caused by: HDFS-15483 Ordered snapshot deletion: Disallow rename between two snapshottable directories
• links to: GitHub Pull Request #4051