Hadoop Common
  1. Hadoop Common
  2. HADOOP-454

hadoop du optionally behave like unix's du -s

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.9.1
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      In Unix, the user has the option to specify a -s flag:
      -s Display an entry for each specified file. (Equivalent to -d 0)

      Instead of providing data for the subdirectories of the arguments, du prints the total usage of each argument, without printing subdirectories.

      This is very convenient for those looking to find how much space is used by a particular branch, without requring them to add up all the sub-usages.

      1. dus.patch
        2 kB
        Hairong Kuang
      2. dus_new.patch
        3 kB
        Hairong Kuang

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        120d 6h 42m 1 Hairong Kuang 13/Dec/06 01:46
        Patch Available Patch Available Resolved Resolved
        20h 25m 1 Doug Cutting 13/Dec/06 22:11
        Resolved Resolved Closed Closed
        51d 5h 15m 1 Doug Cutting 03/Feb/07 03:26
        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Nigel Daley added a comment -

        Disregard the -1 comment. The new patch was committed before my automated process tested it, thus the patch command failed.

        Show
        Nigel Daley added a comment - Disregard the -1 comment. The new patch was committed before my automated process tested it, thus the patch command failed.
        Hide
        Hadoop QA added a comment -

        -1, since the patch command could not apply the latest attachment as a patch: http://issues.apache.org/jira/secure/attachment/12347118/dus_new.patch. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

        Show
        Hadoop QA added a comment - -1, since the patch command could not apply the latest attachment as a patch: http://issues.apache.org/jira/secure/attachment/12347118/dus_new.patch . Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
        Doug Cutting made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Hairong!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Hairong!
        Hairong Kuang made changes -
        Attachment dus_new.patch [ 12347118 ]
        Hide
        Hairong Kuang added a comment -

        The updated patch also includes the changes to the dfs shell usage messages.

        Show
        Hairong Kuang added a comment - The updated patch also includes the changes to the dfs shell usage messages.
        Hide
        Hadoop QA added a comment -

        +1, http://issues.apache.org/jira/secure/attachment/12347055/dus.patch applied and successfully tested against trunk revision r486399

        Show
        Hadoop QA added a comment - +1, http://issues.apache.org/jira/secure/attachment/12347055/dus.patch applied and successfully tested against trunk revision r486399
        Hairong Kuang made changes -
        Fix Version/s 0.10.0 [ 12312207 ]
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.9.1 [ 12312214 ]
        Affects Version/s 0.5.0 [ 12311939 ]
        Hairong Kuang made changes -
        Attachment dus.patch [ 12347055 ]
        Hide
        Hairong Kuang added a comment -

        This patch adds a command "dus" to dfs shell, which behaves like Unix "du -s".

        Show
        Hairong Kuang added a comment - This patch adds a command "dus" to dfs shell, which behaves like Unix "du -s".
        Hide
        Hairong Kuang added a comment -

        I plan to solve the problem by introducing a new command "dus", which prints the total disk usage for each argument. This solution is aligned to what we have in the dfs shell: "lsr" for "ls -r" and "rmr" for "rm -r".

        A better and a long term solution is to allow options for dfs commands, i.e., supporting "du -s" instead of "dus". But we need to enhance dfs command arguments parsing to handle options. Something like common-cli is a good choice.

        Show
        Hairong Kuang added a comment - I plan to solve the problem by introducing a new command "dus", which prints the total disk usage for each argument. This solution is aligned to what we have in the dfs shell: "lsr" for "ls -r" and "rmr" for "rm -r". A better and a long term solution is to allow options for dfs commands, i.e., supporting "du -s" instead of "dus". But we need to enhance dfs command arguments parsing to handle options. Something like common-cli is a good choice.
        Hairong Kuang made changes -
        Field Original Value New Value
        Assignee Hairong Kuang [ hairong ]
        Marco Nicosia created issue -

          People

          • Assignee:
            Hairong Kuang
            Reporter:
            Marco Nicosia
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development