Hadoop Common
  1. Hadoop Common
  2. HADOOP-8710

Remove ability for users to easily run the trash emptier

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      The trash emptier may no longer be run using "hadoop org.apache.hadoop.fs.Trash". The trash emptier runs on the NameNode (if configured). Old trash checkpoints may be deleted using "hadoop fs -expunge".

      Description

      Users can currently run the emptier via hadoop org.apache.hadoop.fs.Trash, which seems error prone as there's nothing in that command that suggests it runs the emptier and nothing that asks you before deleting the trash for all users (that the current user is capable of deleting). Given that the trash emptier runs server side (eg on the NN) let's remove the ability to easily run it client side. Marking as an incompatible change since someone expecting the hadoop command with this class specified to empty trash will no longer be able to (they'll need to create their own class that does this).

      1. hadoop-8710.txt
        0.8 kB
        Eli Collins

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          Patch attached.

          Show
          Eli Collins added a comment - Patch attached.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12541405/hadoop-8710.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1326//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1326//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12541405/hadoop-8710.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1326//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1326//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          +1, the patch looks good to me.

          Show
          Aaron T. Myers added a comment - +1, the patch looks good to me.
          Hide
          Eli Collins added a comment -

          Btw ATM pointed out off list that users can still use the shell's expunge command to run part of the emptier functionality (TrashPolicy#deleteCheckpoint).

          Show
          Eli Collins added a comment - Btw ATM pointed out off list that users can still use the shell's expunge command to run part of the emptier functionality (TrashPolicy#deleteCheckpoint).
          Hide
          Eli Collins added a comment -

          Thanks for the review ATM. I've committed this and merged to branch-2.

          Show
          Eli Collins added a comment - Thanks for the review ATM. I've committed this and merged to branch-2.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2665 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2665/)
          HADOOP-8710. Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2665 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2665/ ) HADOOP-8710 . Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2601 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2601/)
          HADOOP-8710. Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2601 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2601/ ) HADOOP-8710 . Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2630 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2630/)
          HADOOP-8710. Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2630 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2630/ ) HADOOP-8710 . Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1138 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1138/)
          HADOOP-8710. Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1138 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1138/ ) HADOOP-8710 . Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1170 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1170/)
          HADOOP-8710. Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1170 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1170/ ) HADOOP-8710 . Remove ability for users to easily run the trash emptire. Contributed by Eli Collins (Revision 1374476) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374476 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java

            People

            • Assignee:
              Eli Collins
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development