Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13150

Avoid use of toString() in output of HDFS ACL shell commands.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The HDFS ACL shell commands have at least one usage of the standard Java toString method to generate shell output (AclEntry#toString). This issue tracks conversion of that code to use methods other than toString. The toString method is useful primarily for debugging. It's preferrable to use a different method to ensure stable output for public interfaces, such as the shell.

      1. HADOOP-13150.001.patch
        9 kB
        Chris Nauroth
      2. HADOOP-13150.002.patch
        9 kB
        Chris Nauroth

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10560 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10560/)
        HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell (cnauroth: rev 1d330fbaf6b50802750aa461640773fb788ef884)

        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/AclCommands.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclStatus.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntryScope.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AclPermissionParam.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntryType.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10560 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10560/ ) HADOOP-13150 . Avoid use of toString() in output of HDFS ACL shell (cnauroth: rev 1d330fbaf6b50802750aa461640773fb788ef884) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/AclCommands.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclStatus.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntryScope.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AclPermissionParam.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntry.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/AclEntryType.java
        Hide
        cnauroth Chris Nauroth added a comment -

        Steve, thank you for reviewing. I have committed this to trunk, branch-2 and branch-2.8.

        Show
        cnauroth Chris Nauroth added a comment - Steve, thank you for reviewing. I have committed this to trunk, branch-2 and branch-2.8.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        LGTM: +1

        Show
        stevel@apache.org Steve Loughran added a comment - LGTM: +1
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 7m 59s trunk passed
        +1 compile 7m 43s trunk passed
        +1 checkstyle 1m 29s trunk passed
        +1 mvnsite 2m 35s trunk passed
        +1 mvneclipse 0m 41s trunk passed
        +1 findbugs 4m 43s trunk passed
        +1 javadoc 2m 12s trunk passed
        0 mvndep 0m 18s Maven dependency ordering for patch
        +1 mvninstall 2m 28s the patch passed
        +1 compile 7m 35s the patch passed
        +1 javac 7m 35s the patch passed
        +1 checkstyle 1m 30s the patch passed
        +1 mvnsite 2m 44s the patch passed
        +1 mvneclipse 0m 51s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 26s the patch passed
        +1 javadoc 2m 23s the patch passed
        -1 unit 8m 10s hadoop-common in the patch failed.
        +1 unit 1m 3s hadoop-hdfs-client in the patch passed.
        -1 unit 62m 36s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 32s The patch does not generate ASF License warnings.
        146m 45s



        Reason Tests
        Failed junit tests hadoop.net.TestDNS
          hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.datanode.TestFsDatasetCache



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HADOOP-13150
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831789/HADOOP-13150.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5ad467a8b65a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8867762
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 59s trunk passed +1 compile 7m 43s trunk passed +1 checkstyle 1m 29s trunk passed +1 mvnsite 2m 35s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 4m 43s trunk passed +1 javadoc 2m 12s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 2m 28s the patch passed +1 compile 7m 35s the patch passed +1 javac 7m 35s the patch passed +1 checkstyle 1m 30s the patch passed +1 mvnsite 2m 44s the patch passed +1 mvneclipse 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 26s the patch passed +1 javadoc 2m 23s the patch passed -1 unit 8m 10s hadoop-common in the patch failed. +1 unit 1m 3s hadoop-hdfs-client in the patch passed. -1 unit 62m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 146m 45s Reason Tests Failed junit tests hadoop.net.TestDNS   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.datanode.TestFsDatasetCache Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831789/HADOOP-13150.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5ad467a8b65a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8867762 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10677/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        I'm attaching patch 002, just fixing a few typos in JavaDocs that I spotted in the 001 patch.

        Yongjun Zhang, Mingliang Liu and Steve Loughran, you had participated in the code review on HDFS-9732, which was a similar issue about transitioning to a version-stable string output in a different part of the codebase. Are any of you interested in reviewing HADOOP-13150?

        Show
        cnauroth Chris Nauroth added a comment - I'm attaching patch 002, just fixing a few typos in JavaDocs that I spotted in the 001 patch. Yongjun Zhang , Mingliang Liu and Steve Loughran , you had participated in the code review on HDFS-9732 , which was a similar issue about transitioning to a version-stable string output in a different part of the codebase. Are any of you interested in reviewing HADOOP-13150 ?
        Hide
        cnauroth Chris Nauroth added a comment -

        The test failure is unrelated.

        Show
        cnauroth Chris Nauroth added a comment - The test failure is unrelated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 8m 18s trunk passed
        +1 compile 8m 13s trunk passed
        +1 checkstyle 1m 33s trunk passed
        +1 mvnsite 2m 39s trunk passed
        +1 mvneclipse 0m 43s trunk passed
        +1 findbugs 5m 25s trunk passed
        +1 javadoc 2m 27s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 2m 16s the patch passed
        +1 compile 8m 12s the patch passed
        +1 javac 8m 12s the patch passed
        +1 checkstyle 1m 37s the patch passed
        +1 mvnsite 2m 59s the patch passed
        +1 mvneclipse 0m 52s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 54s the patch passed
        +1 javadoc 2m 31s the patch passed
        -1 unit 8m 38s hadoop-common in the patch failed.
        +1 unit 1m 2s hadoop-hdfs-client in the patch passed.
        +1 unit 62m 49s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 31s The patch does not generate ASF License warnings.
        151m 9s



        Reason Tests
        Failed junit tests hadoop.ha.TestZKFailoverController



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HADOOP-13150
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831130/HADOOP-13150.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 057bdfeb4e2b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0670149
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 8m 18s trunk passed +1 compile 8m 13s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 2m 39s trunk passed +1 mvneclipse 0m 43s trunk passed +1 findbugs 5m 25s trunk passed +1 javadoc 2m 27s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 16s the patch passed +1 compile 8m 12s the patch passed +1 javac 8m 12s the patch passed +1 checkstyle 1m 37s the patch passed +1 mvnsite 2m 59s the patch passed +1 mvneclipse 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 54s the patch passed +1 javadoc 2m 31s the patch passed -1 unit 8m 38s hadoop-common in the patch failed. +1 unit 1m 2s hadoop-hdfs-client in the patch passed. +1 unit 62m 49s hadoop-hdfs in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 151m 9s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831130/HADOOP-13150.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 057bdfeb4e2b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0670149 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10634/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        I'm attaching patch 001. This is similar to what was done for HDFS-9732.

        • I have defined toStringStable methods and updated call sites in the shell and WebHDFS.
        • I took the opportunity to update several of the ACL classes from Evolving to Stable. The API hasn't changed since initial release, so I'm confident that we can declare it stable.
        • AclPermissionParam#parseAclSpec needed to stop calling StringUtils#join, because it internally calls toString. The new logic is an adaptation of the logic of StringUtils#join.
        • I have not added any new tests. The logic is already covered well by existing tests that parse ACL strings back and forth.
        Show
        cnauroth Chris Nauroth added a comment - I'm attaching patch 001. This is similar to what was done for HDFS-9732 . I have defined toStringStable methods and updated call sites in the shell and WebHDFS. I took the opportunity to update several of the ACL classes from Evolving to Stable . The API hasn't changed since initial release, so I'm confident that we can declare it stable. AclPermissionParam#parseAclSpec needed to stop calling StringUtils#join , because it internally calls toString . The new logic is an adaptation of the logic of StringUtils#join . I have not added any new tests. The logic is already covered well by existing tests that parse ACL strings back and forth.

          People

          • Assignee:
            cnauroth Chris Nauroth
            Reporter:
            cnauroth Chris Nauroth
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development