Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.3.0
    • Component/s: None
    • Labels:
      None

      Description

      Added ability to set replication for a directory/file from the command line.
      Not heavily tested...

      1. setrep-v2.patch
        4 kB
        Johan Oskarsson
      2. setrep.patch
        3 kB
        Johan Oskarsson

        Activity

        Hide
        Konstantin Shvachko added a comment -

        This is very useful.
        Some things that I have noticed.

        • Usage info might be useful.
          Right now it just throws ArrayIndexOutOfBoundsException if you do not specify the path.
          Or some other exception if you forget to provide the replication number. Try
          hadoop dfs -setrep
          hadoop dfs -setrep 3
          hadoop dfs -setrep /user
        • I think the 2 commands should be combined into one like
          hadoop dfs -setrep [-R] <path>
        • Should we use abbreviations like "setrep" or full names?
          see HADOOP-226 (4) about necessity for a general DFSShell convention.
        Show
        Konstantin Shvachko added a comment - This is very useful. Some things that I have noticed. Usage info might be useful. Right now it just throws ArrayIndexOutOfBoundsException if you do not specify the path. Or some other exception if you forget to provide the replication number. Try hadoop dfs -setrep hadoop dfs -setrep 3 hadoop dfs -setrep /user I think the 2 commands should be combined into one like hadoop dfs -setrep [-R] <path> Should we use abbreviations like "setrep" or full names? see HADOOP-226 (4) about necessity for a general DFSShell convention.
        Hide
        Johan Oskarsson added a comment -

        Yeah, I know there are issues like that, this is mostly a quick hack I needed to set the replication in order to save space.

        My question is, should I fix them or should I wait for a complete DFSShell rehaul with Commons CLI?
        I currently do not have time to rewrite the DFSShell myself, but perhaps if there's been no attempt made in a few weeks I might give it a try.

        You're absolutely right that they should be combined into one, also related to the whole issue with proper command line parsing etc.
        Same thing with -ls and -lsr I suppose.

        Thanks for the comments.

        Show
        Johan Oskarsson added a comment - Yeah, I know there are issues like that, this is mostly a quick hack I needed to set the replication in order to save space. My question is, should I fix them or should I wait for a complete DFSShell rehaul with Commons CLI? I currently do not have time to rewrite the DFSShell myself, but perhaps if there's been no attempt made in a few weeks I might give it a try. You're absolutely right that they should be combined into one, also related to the whole issue with proper command line parsing etc. Same thing with -ls and -lsr I suppose. Thanks for the comments.
        Hide
        Andrzej Bialecki added a comment -

        Other things that I would love to see added to DFSShell (to make it more shell-like). They are not such a big deal to implement, but they would increase the usefulness of this class tremendously (note: this is orthogonal to the issue of using Commons CLI, which I'd love to see too):

        • globbing and multi-argument cp/mv/rm/ls, using the standard shell metacharacters
        • interactive mode: ability to issue several commands in the same session, while keeping track of the context (such as pwd)
        • scripts: nothing fancy, just the ability to execute several commands recorded in a text file.

        The scriptability and interactive mode, and a host of other useful features we can throw in cheaply by using Rhino (http://www.mozilla.org/rhino/shell.html).

        Show
        Andrzej Bialecki added a comment - Other things that I would love to see added to DFSShell (to make it more shell-like). They are not such a big deal to implement, but they would increase the usefulness of this class tremendously (note: this is orthogonal to the issue of using Commons CLI, which I'd love to see too): globbing and multi-argument cp/mv/rm/ls, using the standard shell metacharacters interactive mode: ability to issue several commands in the same session, while keeping track of the context (such as pwd) scripts: nothing fancy, just the ability to execute several commands recorded in a text file. The scriptability and interactive mode, and a host of other useful features we can throw in cheaply by using Rhino ( http://www.mozilla.org/rhino/shell.html ).
        Hide
        Konstantin Shvachko added a comment -

        Johan, I would vote for your patch if you fix it.

        Show
        Konstantin Shvachko added a comment - Johan, I would vote for your patch if you fix it.
        Hide
        Johan Oskarsson added a comment -

        I finally had some time to fix the issues in the previous patch.
        Posting new patch that shouldn't throw any nasty exceptions if the user doesn't provide all the needed input

        Show
        Johan Oskarsson added a comment - I finally had some time to fix the issues in the previous patch. Posting new patch that shouldn't throw any nasty exceptions if the user doesn't provide all the needed input
        Hide
        Johan Oskarsson added a comment -

        New version with error handling etc

        Show
        Johan Oskarsson added a comment - New version with error handling etc
        Hide
        Konstantin Shvachko added a comment -

        +1

        Show
        Konstantin Shvachko added a comment - +1
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Johan!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Johan!

          People

          • Assignee:
            Johan Oskarsson
            Reporter:
            Johan Oskarsson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development