Hadoop Common
  1. Hadoop Common
  2. HADOOP-454

hadoop du optionally behave like unix's du -s

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.9.1
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      In Unix, the user has the option to specify a -s flag:
      -s Display an entry for each specified file. (Equivalent to -d 0)

      Instead of providing data for the subdirectories of the arguments, du prints the total usage of each argument, without printing subdirectories.

      This is very convenient for those looking to find how much space is used by a particular branch, without requring them to add up all the sub-usages.

      1. dus.patch
        2 kB
        Hairong Kuang
      2. dus_new.patch
        3 kB
        Hairong Kuang

        Activity

        Hide
        Hairong Kuang added a comment -

        I plan to solve the problem by introducing a new command "dus", which prints the total disk usage for each argument. This solution is aligned to what we have in the dfs shell: "lsr" for "ls -r" and "rmr" for "rm -r".

        A better and a long term solution is to allow options for dfs commands, i.e., supporting "du -s" instead of "dus". But we need to enhance dfs command arguments parsing to handle options. Something like common-cli is a good choice.

        Show
        Hairong Kuang added a comment - I plan to solve the problem by introducing a new command "dus", which prints the total disk usage for each argument. This solution is aligned to what we have in the dfs shell: "lsr" for "ls -r" and "rmr" for "rm -r". A better and a long term solution is to allow options for dfs commands, i.e., supporting "du -s" instead of "dus". But we need to enhance dfs command arguments parsing to handle options. Something like common-cli is a good choice.
        Hide
        Hairong Kuang added a comment -

        This patch adds a command "dus" to dfs shell, which behaves like Unix "du -s".

        Show
        Hairong Kuang added a comment - This patch adds a command "dus" to dfs shell, which behaves like Unix "du -s".
        Hide
        Hadoop QA added a comment -

        +1, http://issues.apache.org/jira/secure/attachment/12347055/dus.patch applied and successfully tested against trunk revision r486399

        Show
        Hadoop QA added a comment - +1, http://issues.apache.org/jira/secure/attachment/12347055/dus.patch applied and successfully tested against trunk revision r486399
        Hide
        Hairong Kuang added a comment -

        The updated patch also includes the changes to the dfs shell usage messages.

        Show
        Hairong Kuang added a comment - The updated patch also includes the changes to the dfs shell usage messages.
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Hairong!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Hairong!
        Hide
        Hadoop QA added a comment -

        -1, since the patch command could not apply the latest attachment as a patch: http://issues.apache.org/jira/secure/attachment/12347118/dus_new.patch. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

        Show
        Hadoop QA added a comment - -1, since the patch command could not apply the latest attachment as a patch: http://issues.apache.org/jira/secure/attachment/12347118/dus_new.patch . Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
        Hide
        Nigel Daley added a comment -

        Disregard the -1 comment. The new patch was committed before my automated process tested it, thus the patch command failed.

        Show
        Nigel Daley added a comment - Disregard the -1 comment. The new patch was committed before my automated process tested it, thus the patch command failed.

          People

          • Assignee:
            Hairong Kuang
            Reporter:
            Marco Nicosia
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development