Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6620

Snapshot docs should specify about preserve options with cp command

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.5.0
    • Fix Version/s: 2.5.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently snapshot docs contains the cp command details for copying the snapshot files to regular dfs tree.

      http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html#Snapshot_Paths

      Copying a file from snapshot s0:
      hdfs dfs -cp /foo/.snapshot/s0/bar /tmp

      Since there are important properties(acls, xattrs..etc) for a file may need to copy back. But looking at this doc may mislead to copy with out using any options. So, It may be worth mentioning aboit that properties. Infact it should copy all properties by default from snapshot file.

      1. HDFS-6620.001.patch
        3 kB
        Stephen Chu
      2. HDFS-6620.002.patch
        3 kB
        Stephen Chu

        Activity

        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1821 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1821/)
        HDFS-6620. Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1821 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1821/ ) HDFS-6620 . Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #1794 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1794/)
        HDFS-6620. Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1794 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1794/ ) HDFS-6620 . Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #603 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/603/)
        HDFS-6620. Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #603 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/603/ ) HDFS-6620 . Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Hide
        Stephen Chu added a comment -

        Thank you, Uma Maheswara Rao G!

        Show
        Stephen Chu added a comment - Thank you, Uma Maheswara Rao G !
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12653856/HDFS-6620.002.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7280//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7280//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653856/HDFS-6620.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7280//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7280//console This message is automatically generated.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5822 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5822/)
        HDFS-6620. Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5822 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5822/ ) HDFS-6620 . Snapshot docs should specify about preserve options with cp command. Contributed by Stephen Chu. (umamahesh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1607685 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/xdoc/HdfsSnapshots.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestXAttrWithSnapshot.java
        Hide
        Uma Maheswara Rao G added a comment -

        I have committed this to trunk and branch-2. Thanks a lot Stephen.

        Show
        Uma Maheswara Rao G added a comment - I have committed this to trunk and branch-2. Thanks a lot Stephen.
        Hide
        Uma Maheswara Rao G added a comment -

        +1 patch looks good to me. I will commit the patch shortly.

        Show
        Uma Maheswara Rao G added a comment - +1 patch looks good to me. I will commit the patch shortly.
        Hide
        Stephen Chu added a comment -

        Thanks for the review, Uma Maheswara Rao G. Sounds good. Updated a new patch that addresses your comments.

        Show
        Stephen Chu added a comment - Thanks for the review, Uma Maheswara Rao G . Sounds good. Updated a new patch that addresses your comments.
        Hide
        Uma Maheswara Rao G added a comment -

        Thanks a lot for the patch Stephen Chu.
        Overall patch looks good to me though I have few nits to address.

        ACLs, and XAttrs --> ACLs and XAttrs
        Please remove comma.

        testCopyPreserveSnapshotXAttrs --> testCopySnapshotShouldPreserveXattrs ?

        Show
        Uma Maheswara Rao G added a comment - Thanks a lot for the patch Stephen Chu . Overall patch looks good to me though I have few nits to address. ACLs, and XAttrs --> ACLs and XAttrs Please remove comma. testCopyPreserveSnapshotXAttrs --> testCopySnapshotShouldPreserveXattrs ?
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12653654/HDFS-6620.001.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7273//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7273//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653654/HDFS-6620.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/7273//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7273//console This message is automatically generated.
        Hide
        Stephen Chu added a comment -

        Attaching a patch that

        • changes the cp snapshot doc example to use the -ptopax option. I added a note that the example shows how to preserve timestamp, ownership, permissions, acls, and xattrs.
        • Added an additional XAttr + Snapshot unit test to verify that users can copy a snapshot file and preserve its attributes.
        Show
        Stephen Chu added a comment - Attaching a patch that changes the cp snapshot doc example to use the -ptopax option. I added a note that the example shows how to preserve timestamp, ownership, permissions, acls, and xattrs. Added an additional XAttr + Snapshot unit test to verify that users can copy a snapshot file and preserve its attributes.
        Hide
        Stephen Chu added a comment -

        Good idea, I can take this up. Talked to Uma who suggested that we also add a test case for copying a snapshot file containing xattrs back to the main dfs tree and verifying that the xattrs are retained.

        Show
        Stephen Chu added a comment - Good idea, I can take this up. Talked to Uma who suggested that we also add a test case for copying a snapshot file containing xattrs back to the main dfs tree and verifying that the xattrs are retained.

          People

          • Assignee:
            Stephen Chu
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development