Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3302

Review and improve HDFS trash documentation

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: documentation
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Improve Trash documentation for users.

      (0.23 published release docs are missing original HDFS docs btw...)

      A set of FAQ-like questions can be found on HDFS-2740

      I'll update the ticket shortly with the areas to cover in the docs, as enabling trash by default (HDFS-2740) would be considered as a wide behavior change per its follow ups.

        Issue Links

          Activity

          Hide
          bitti David Ongaro added a comment -

          Why is this "Major" priority and still open? The current documentation reads like the trash feature is enabled by default. I think we could save some lives here by clarifying this.

          That and the fact that you denied a confirmation message for "hadoop fs -rm/rmr" because there "is" a trash feature is a deadly combination.

          Show
          bitti David Ongaro added a comment - Why is this "Major" priority and still open? The current documentation reads like the trash feature is enabled by default. I think we could save some lives here by clarifying this. That and the fact that you denied a confirmation message for "hadoop fs -rm/rmr" because there "is" a trash feature is a deadly combination.
          Hide
          pmkiran19 Madhu Kiran added a comment -

          Added more information related to HDFS Trash documentation

          Show
          pmkiran19 Madhu Kiran added a comment - Added more information related to HDFS Trash documentation
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 2m 52s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
          +1 site 2m 53s Site still builds.
          -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix.
              6m 10s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731454/HDFS-3302.patch
          Optional Tests site
          git revision trunk / 6f62267
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10871/artifact/patchprocess/whitespace.txt
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10871/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 2m 52s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 site 2m 53s Site still builds. -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix.     6m 10s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731454/HDFS-3302.patch Optional Tests site git revision trunk / 6f62267 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/10871/artifact/patchprocess/whitespace.txt Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10871/console This message was automatically generated.
          Hide
          iwasakims Masatake Iwasaki added a comment -

          Thanks for the patch, Madhu Kiran.

          I think examples of command lines and outputs is too specific and verbose for Architecture doc. How about just emphasizing that trash folder is disabled by default in "File Deletes and Undeletes" section? For example,

          - When a file is deleted by a user or an application, it is not immediately removed from HDFS. ...
          + If you enabled trash directory, a file deleted by a user or an application is not immediately removed from HDFS. ...
          

          Also, a link to Architecture Guide in the usage of -expunge command in "FileSystem Shell" document could be more specific.

          -Empty the Trash. Refer to the [HDFS Architecture Guide](../hadoop-hdfs/HdfsDesign.html) for ...
          +Empty the Trash. Refer to the [HDFS Architecture Guide](../hadoop-hdfs/HdfsDesign.html#File_Deletes_and_Undeletes) for ...
          

          It could be good to add same notice to the usage of -rm command too.

          Show
          iwasakims Masatake Iwasaki added a comment - Thanks for the patch, Madhu Kiran . I think examples of command lines and outputs is too specific and verbose for Architecture doc. How about just emphasizing that trash folder is disabled by default in "File Deletes and Undeletes" section ? For example, - When a file is deleted by a user or an application, it is not immediately removed from HDFS. ... + If you enabled trash directory, a file deleted by a user or an application is not immediately removed from HDFS. ... Also, a link to Architecture Guide in the usage of -expunge command in "FileSystem Shell" document could be more specific. -Empty the Trash. Refer to the [HDFS Architecture Guide](../hadoop-hdfs/HdfsDesign.html) for ... +Empty the Trash. Refer to the [HDFS Architecture Guide](../hadoop-hdfs/HdfsDesign.html#File_Deletes_and_Undeletes) for ... It could be good to add same notice to the usage of -rm command too.
          Hide
          cheersyang Weiwei Yang added a comment -

          I also find the trash document is confusing, I have created a patch in HADOOP-5323, to improve the documentation in http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#File_Deletes_and_Undeletes.

          Appreciate folks to review. Thanks.

          Show
          cheersyang Weiwei Yang added a comment - I also find the trash document is confusing, I have created a patch in HADOOP-5323 , to improve the documentation in http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#File_Deletes_and_Undeletes . Appreciate folks to review. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 3m 12s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 site 2m 57s Site still builds.
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
              6m 34s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731454/HDFS-3302.patch
          Optional Tests site
          git revision trunk / 095ab9a
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12246/artifact/patchprocess/whitespace.txt
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12246/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 3m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 site 2m 57s Site still builds. -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.     6m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731454/HDFS-3302.patch Optional Tests site git revision trunk / 095ab9a whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12246/artifact/patchprocess/whitespace.txt Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12246/console This message was automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          +1. Committing.

          Show
          wheat9 Haohui Mai added a comment - +1. Committing.
          Hide
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk and branch-2. Thanks Harsh J for the contribution.

          Show
          wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Harsh J for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8845 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8845/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8845 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8845/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2637 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2637/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2637 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2637/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #697 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/697/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #697 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/697/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #708 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/708/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #708 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/708/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1433 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1433/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1433 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1433/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2567 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2567/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2567 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2567/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/629/)
          HDFS-3302. Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36)

          • hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/629/ ) HDFS-3302 . Review and improve HDFS trash documentation. Contributed by (wheat9: rev 2326171ea84b9ccea9df9fef137d6041df540d36) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
          Hide
          qwertymaniac Harsh J added a comment -

          Just a short note: This seems to be incorrectly attributed (to me). Actual patch contributor was Madhu Kiran.

          Show
          qwertymaniac Harsh J added a comment - Just a short note: This seems to be incorrectly attributed (to me). Actual patch contributor was Madhu Kiran .

            People

            • Assignee:
              Unassigned
              Reporter:
              qwertymaniac Harsh J
            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development