Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2802 Support for RW/RO snapshots in HDFS
  3. HDFS-4514

Add CLI support for snapshot rename, diff report, and snapshottable directory listing

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Snapshot (HDFS-2802)
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. HDFS-4514.003.patch
      24 kB
      Jing Zhao
    2. HDFS-4514.002.patch
      24 kB
      Jing Zhao
    3. Screen Shot 2013-02-22 at 5.38.41 PM.png
      645 kB
      Jing Zhao
    4. HDFS-4514.001.patch
      22 kB
      Jing Zhao
    5. HDFS-4514.000.patch
      15 kB
      Jing Zhao

      Activity

      Hide
      Jing Zhao added a comment -

      Initial patch without testing.

      Show
      Jing Zhao added a comment - Initial patch without testing.
      Hide
      Jing Zhao added a comment -

      Update the patch.

      Show
      Jing Zhao added a comment - Update the patch.
      Hide
      Jing Zhao added a comment -

      screenshot when testing the commands.

      Show
      Jing Zhao added a comment - screenshot when testing the commands.
      Hide
      Tsz Wo Nicholas Sze added a comment -
      • The commands should consistently use either snapshotRoot or snapshotDir. How about using snapshotDir?
      • Do not use assert in RenameSnapshot.processArguments(..). It should throw an exception for illegal arguments.
      • In SnapshottableDirectoryStatus.print(..), nulls should be printed as empty strings.
      • Revert the following:
        @@ -267,7 +267,7 @@ public String getCommandName() {
             public void run(Path path) throws IOException {
               dfs.setQuota(path, HdfsConstants.QUOTA_DONT_SET, quota);
             }
        -  }
        +  }  
           
           /**
        
      • In LsSnapshottableDir,
        • use FileSystem.get(conf) to get default FileSystem;
        • add javadoc.
      Show
      Tsz Wo Nicholas Sze added a comment - The commands should consistently use either snapshotRoot or snapshotDir. How about using snapshotDir? Do not use assert in RenameSnapshot.processArguments(..). It should throw an exception for illegal arguments. In SnapshottableDirectoryStatus.print(..), nulls should be printed as empty strings. Revert the following: @@ -267,7 +267,7 @@ public String getCommandName() { public void run(Path path) throws IOException { dfs.setQuota(path, HdfsConstants.QUOTA_DONT_SET, quota); } - } + } /** In LsSnapshottableDir, use FileSystem.get(conf) to get default FileSystem; add javadoc.
      Hide
      Jing Zhao added a comment -

      Thanks for the comments Nicholas! Update the patch based on your comments.

      In SnapshottableDirectoryStatus.print(..), nulls should be printed as empty strings

      I re-checked the code and it seems that all the fields in the SnapshottableDirectoryStatus for printing should be non-null.

      Show
      Jing Zhao added a comment - Thanks for the comments Nicholas! Update the patch based on your comments. In SnapshottableDirectoryStatus.print(..), nulls should be printed as empty strings I re-checked the code and it seems that all the fields in the SnapshottableDirectoryStatus for printing should be non-null.
      Hide
      Tsz Wo Nicholas Sze added a comment -

      +1 patch looks good.

      Show
      Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
      Hide
      Tsz Wo Nicholas Sze added a comment -

      I have committed this. Thanks, Jing!

      Show
      Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Jing!
      Hide
      Hudson added a comment -

      Integrated in Hadoop-Hdfs-Snapshots-Branch-build #113 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/113/)
      HDFS-4514. Add CLI for supporting snapshot rename, diff report, and snapshottable directory listing. Contributed by Jing Zhao (Revision 1449956)

      Result = FAILURE
      szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1449956
      Files :

      • /hadoop/common/branches/HDFS-2802/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
      • /hadoop/common/branches/HDFS-2802/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/SnapshotCommands.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotDiffReport.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshottableDirectoryStatus.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/LsSnapshottableDir.java
      • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/SnapshotDiff.java
      Show
      Hudson added a comment - Integrated in Hadoop-Hdfs-Snapshots-Branch-build #113 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/113/ ) HDFS-4514 . Add CLI for supporting snapshot rename, diff report, and snapshottable directory listing. Contributed by Jing Zhao (Revision 1449956) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1449956 Files : /hadoop/common/branches/ HDFS-2802 /hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/SnapshotCommands.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-2802 .txt /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotDiffReport.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshottableDirectoryStatus.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/LsSnapshottableDir.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/snapshot/SnapshotDiff.java

        People

        • Assignee:
          Jing Zhao
          Reporter:
          Jing Zhao
        • Votes:
          0 Vote for this issue
          Watchers:
          4 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development