Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11803

Add -v option for du command to show header line

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Like hdfs -count command, it's better to add -v for du command to show header line.

      Without -v,
      $ hdfs -du -h -s /tmp/parent

      1 G  1 G  /tmp/parent
      

      With -v,
      $ hdfs -du -h -s -v /tmp/parent

      SIZE DISK_SPACE_CONSUMED_WITH_ALL_REPLICAS FULL_PATH_NAME
      1 G   1 G        /tmp/parent
      

      $ hdfs dfs -count -q -v -h -x /tmp/parent

      QUOTA       REM_QUOTA     SPACE_QUOTA REM_SPACE_QUOTA    DIR_COUNT   FILE_COUNT       CONTENT_SIZE PATHNAME
      10               7            50 G            49 G            2            1                1 G /tmp/parent
      
      1. HDFS-11803.002.patch
        5 kB
        Xiaobing Zhou
      2. HDFS-11803.001.patch
        5 kB
        Xiaobing Zhou
      3. HDFS-11803.000.patch
        3 kB
        Xiaobing Zhou

        Issue Links

          Activity

          Hide
          xiaobingo Xiaobing Zhou added a comment -

          Posted a patch for review.

          Show
          xiaobingo Xiaobing Zhou added a comment - Posted a patch for review.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          v1 fixes documentation.

          Show
          xiaobingo Xiaobing Zhou added a comment - v1 fixes documentation.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 13m 46s trunk passed
          +1 compile 17m 49s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 1m 7s trunk passed
          +1 mvneclipse 0m 21s trunk passed
          -1 findbugs 1m 26s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
          +1 javadoc 0m 51s trunk passed
          +1 mvninstall 0m 43s the patch passed
          +1 compile 17m 32s the patch passed
          +1 javac 17m 32s the patch passed
          -0 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31)
          +1 mvnsite 1m 3s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 36s the patch passed
          +1 javadoc 0m 50s the patch passed
          -1 unit 8m 50s hadoop-common in the patch failed.
          +1 asflicense 0m 38s The patch does not generate ASF License warnings.
          69m 36s



          Reason Tests
          Failed junit tests hadoop.cli.TestCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11803
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868178/HDFS-11803.000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 38c63fcf454b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c48f297
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19439/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19439/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 46s trunk passed +1 compile 17m 49s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 21s trunk passed -1 findbugs 1m 26s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. +1 javadoc 0m 51s trunk passed +1 mvninstall 0m 43s the patch passed +1 compile 17m 32s the patch passed +1 javac 17m 32s the patch passed -0 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31) +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 36s the patch passed +1 javadoc 0m 50s the patch passed -1 unit 8m 50s hadoop-common in the patch failed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 69m 36s Reason Tests Failed junit tests hadoop.cli.TestCLI Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11803 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868178/HDFS-11803.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 38c63fcf454b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c48f297 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19439/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19439/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19439/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 14m 47s trunk passed
          +1 compile 17m 37s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 12s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          -1 findbugs 1m 40s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
          +1 javadoc 0m 53s trunk passed
          +1 mvninstall 0m 49s the patch passed
          +1 compile 15m 45s the patch passed
          +1 javac 15m 45s the patch passed
          -0 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31)
          +1 mvnsite 1m 4s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 34s the patch passed
          +1 javadoc 0m 48s the patch passed
          -1 unit 7m 0s hadoop-common in the patch failed.
          +1 asflicense 0m 34s The patch does not generate ASF License warnings.
          66m 56s



          Reason Tests
          Failed junit tests hadoop.cli.TestCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11803
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868187/HDFS-11803.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux acdaf7d11a1e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c48f297
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19440/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19440/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 14m 47s trunk passed +1 compile 17m 37s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 12s trunk passed +1 mvneclipse 0m 19s trunk passed -1 findbugs 1m 40s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. +1 javadoc 0m 53s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 15m 45s the patch passed +1 javac 15m 45s the patch passed -0 checkstyle 0m 36s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31) +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 34s the patch passed +1 javadoc 0m 48s the patch passed -1 unit 7m 0s hadoop-common in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 66m 56s Reason Tests Failed junit tests hadoop.cli.TestCLI Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11803 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868187/HDFS-11803.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux acdaf7d11a1e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c48f297 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19440/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19440/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19440/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vagarychen Chen Liang added a comment -

          Thanks Xiaobing Zhou for the work! v001 patch LGTM, just some NITs about the doc. Will it be better to change "* The -v option displays..." to "* The -v option will display..."? such that it will be consistent with the description of the other options. Also I was thinking of keep the "The du returns three..." part but make the description of -v option simply say that it will display the name of these field as a header line, rather than removing this part and listing the name of fields only in -v option as in the patch.

          Show
          vagarychen Chen Liang added a comment - Thanks Xiaobing Zhou for the work! v001 patch LGTM, just some NITs about the doc. Will it be better to change "* The -v option displays..." to "* The -v option will display..."? such that it will be consistent with the description of the other options. Also I was thinking of keep the "The du returns three..." part but make the description of -v option simply say that it will display the name of these field as a header line, rather than removing this part and listing the name of fields only in -v option as in the patch.
          Hide
          xiaobingo Xiaobing Zhou added a comment -

          v002 fixed the test failure and addressed your review, thanks Chen Liang.

          Show
          xiaobingo Xiaobing Zhou added a comment - v002 fixed the test failure and addressed your review, thanks Chen Liang .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 13m 51s trunk passed
          +1 compile 14m 25s trunk passed
          +1 checkstyle 0m 36s trunk passed
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 20s trunk passed
          -1 findbugs 1m 27s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
          +1 javadoc 0m 54s trunk passed
          +1 mvninstall 0m 43s the patch passed
          +1 compile 13m 36s the patch passed
          +1 javac 13m 36s the patch passed
          -0 checkstyle 0m 37s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31)
          +1 mvnsite 1m 14s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 53s the patch passed
          +1 unit 8m 43s hadoop-common in the patch passed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          62m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11803
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869011/HDFS-11803.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux ae2aa699672b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 19482e0
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19514/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19514/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19514/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19514/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 51s trunk passed +1 compile 14m 25s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 20s trunk passed -1 findbugs 1m 27s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 43s the patch passed +1 compile 13m 36s the patch passed +1 javac 13m 36s the patch passed -0 checkstyle 0m 37s hadoop-common-project/hadoop-common: The patch generated 1 new + 31 unchanged - 0 fixed = 32 total (was 31) +1 mvnsite 1m 14s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 53s the patch passed +1 unit 8m 43s hadoop-common in the patch passed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 62m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11803 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869011/HDFS-11803.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ae2aa699672b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 19482e0 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19514/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19514/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19514/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19514/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          vagarychen Chen Liang added a comment -

          thanks Xiaobing Zhou for v002 patch! LGTM, +1

          Show
          vagarychen Chen Liang added a comment - thanks Xiaobing Zhou for v002 patch! LGTM, +1
          Hide
          liuml07 Mingliang Liu added a comment -

          Committed to trunk. Thanks for your contribution Xiaobing Zhou. Thanks for your review Chen Liang.

          Show
          liuml07 Mingliang Liu added a comment - Committed to trunk . Thanks for your contribution Xiaobing Zhou . Thanks for your review Chen Liang .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11757 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11757/)
          HDFS-11803. Add -v option for du command to show header line. (liuml07: rev 1db186f662af971d956d74aed03150b0692a2315)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/FsUsage.java
          • (edit) hadoop-common-project/hadoop-common/src/test/resources/testConf.xml
          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11757 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11757/ ) HDFS-11803 . Add -v option for du command to show header line. (liuml07: rev 1db186f662af971d956d74aed03150b0692a2315) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/FsUsage.java (edit) hadoop-common-project/hadoop-common/src/test/resources/testConf.xml (edit) hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md

            People

            • Assignee:
              xiaobingo Xiaobing Zhou
              Reporter:
              xiaobingo Xiaobing Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development