Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Document hadoop command line options in forrest. HADOOP-2908 covered the DFS options. This jira is for commands other than the DFS Shell.

      1. commands_manual.pdf
        35 kB
        Sharad Agarwal
      2. commands_manual.pdf
        39 kB
        Sharad Agarwal
      3. 3552_3.patch
        27 kB
        Sharad Agarwal
      4. 3552_2.patch
        25 kB
        Sharad Agarwal
      5. 3552_1.patch
        25 kB
        Sharad Agarwal

        Activity

        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #533 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/533/ )
        Hide
        Chris Douglas added a comment -

        I just committed this. Thanks, Sharad

        Show
        Chris Douglas added a comment - I just committed this. Thanks, Sharad
        Hide
        Konstantin Shvachko added a comment -

        +1

        Show
        Konstantin Shvachko added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12384749/3552_3.patch
        against trunk revision 671563.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12384749/3552_3.patch against trunk revision 671563. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2748/console This message is automatically generated.
        Hide
        Sharad Agarwal added a comment -

        the job option has nothing to do with the libjars and others. It during running the jar that they are useful.

        job -submit does use libjars option. Checked at configureCommandLineOptions in JobClient.

        but is this document supposed to serve as the main doucment and points ...

        I agree there is already some of this information scattered in various other documents in pieces. This is the starting point for this doc. Eventually all command related information in various other documents should get moved to this one. Other documents should talk about the theory behind a feature. For command related info, point to this document.
        However, this evolution will take time and for release 18, we should be good with the basic stuff in place. make sense?

        Show
        Sharad Agarwal added a comment - the job option has nothing to do with the libjars and others. It during running the jar that they are useful. job -submit does use libjars option. Checked at configureCommandLineOptions in JobClient. but is this document supposed to serve as the main doucment and points ... I agree there is already some of this information scattered in various other documents in pieces. This is the starting point for this doc. Eventually all command related information in various other documents should get moved to this one. Other documents should talk about the theory behind a feature. For command related info, point to this document. However, this evolution will take time and for release 18, we should be good with the basic stuff in place. make sense?
        Hide
        Sharad Agarwal added a comment -
        • Incorporated Konstantin's comments.
        • added bin/hadoop archive

          I do not understand what is the idea under partitioning the set of commands into Administration and User groups.

          User commands are supposed to be used by cluster users. Users wont be doing operations like starting a namenode or tasktracker etc, doing rebalancing etc. So these have been grouped into Administration along with dfsadmin.

        Show
        Sharad Agarwal added a comment - Incorporated Konstantin's comments. added bin/hadoop archive I do not understand what is the idea under partitioning the set of commands into Administration and User groups. User commands are supposed to be used by cluster users. Users wont be doing operations like starting a namenode or tasktracker etc, doing rebalancing etc. So these have been grouped into Administration along with dfsadmin.
        Hide
        Konstantin Shvachko added a comment -

        I think this is a good start for the shell command manual.

        • hdfs_user_guide should contain a link to the manual.
        • Could you please replace DFS with HDFS. We were trying to use HDFS instead of DFS at least in new documents.
        • A typo: Ctll-C
        • I do not understand what is the idea under partitioning the set of commands into Administration and User groups.
          All administrator commands are called using dfsadmin so these are "real" administration commands.
          May be you want to change the name of section 2 or just remove this level of section hierarchy making each command
          a section rather than a sub-section.
        Show
        Konstantin Shvachko added a comment - I think this is a good start for the shell command manual. hdfs_user_guide should contain a link to the manual. Could you please replace DFS with HDFS. We were trying to use HDFS instead of DFS at least in new documents. A typo: Ctll-C I do not understand what is the idea under partitioning the set of commands into Administration and User groups. All administrator commands are called using dfsadmin so these are "real" administration commands. May be you want to change the name of section 2 or just remove this level of section hierarchy making each command a section rather than a sub-section.
        Hide
        Mahadev konar added a comment -

        1) the job option has nothing to do with the libjars and others. It during running the jar that they are useful.
        2) and it misses archive

        bin/hadoop archive

        3) sorry to be asking this-- but is this document supposed to serve as the main doucment and points to the others in case of specific information? The problem I see with this is that it is not a complete documentation. If you make changes to any command interface say dfsadmin then you will have to change this document and also the main document for dfsadmin – which seems redundant. We could just have minimal information in thos document and point it to the main document for info

        so info just like this —

        dfsadmin go to the link for dfsadmin
        dfs commands go to link for dfsdommand

        that we do not have to update twice – in this document and the main one.

        Show
        Mahadev konar added a comment - 1) the job option has nothing to do with the libjars and others. It during running the jar that they are useful. 2) and it misses archive bin/hadoop archive 3) sorry to be asking this-- but is this document supposed to serve as the main doucment and points to the others in case of specific information? The problem I see with this is that it is not a complete documentation. If you make changes to any command interface say dfsadmin then you will have to change this document and also the main document for dfsadmin – which seems redundant. We could just have minimal information in thos document and point it to the main document for info so info just like this — dfsadmin go to the link for dfsadmin dfs commands go to link for dfsdommand that we do not have to update twice – in this document and the main one.
        Hide
        Arun C Murthy added a comment -

        I believe that it will be much more useful with usage examples ala HADOOP-2908.

        Show
        Arun C Murthy added a comment - I believe that it will be much more useful with usage examples ala HADOOP-2908 .
        Hide
        Sharad Agarwal added a comment -

        minor typos fixed.

        Show
        Sharad Agarwal added a comment - minor typos fixed.
        Hide
        Sharad Agarwal added a comment -

        Attaching the patch.
        Since I am just a month old kid in hadoop land I have tried to fill in as much details I could; and will let others to fill in more if required.

        Show
        Sharad Agarwal added a comment - Attaching the patch. Since I am just a month old kid in hadoop land I have tried to fill in as much details I could; and will let others to fill in more if required.
        Hide
        Mukund Madhugiri added a comment -

        Marking a blocker for 0.18.0

        Show
        Mukund Madhugiri added a comment - Marking a blocker for 0.18.0

          People

          • Assignee:
            Sharad Agarwal
            Reporter:
            Sharad Agarwal
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development