Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: contrib/streaming
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Here are some of the findings by Karam while verifying -info option in streaming:

      1. We need to add "Optional" for -mapper, -reducer,-combiner and -file options.
      2. For -inputformat and -outputformat options, we should put "Optional" in the prefix for the sake on uniformity.
      3. We need to remove -cluster decription.
      4. -help option is not displayed in usage message.
      5. when displaying message for -info or -help options, we should not display "Streaming Job Failed!"; also exit code should be 0 in case of -help/-info option.
      1. patch-1911-2.txt
        21 kB
        Amareshwari Sriramadasu
      2. patch-1911-1.txt
        20 kB
        Amareshwari Sriramadasu
      3. patch-1911.txt
        16 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
          Hide
          Amareshwari Sriramadasu added a comment -

          I just committed this!

          Show
          Amareshwari Sriramadasu added a comment - I just committed this!
          Hide
          Ravi Gummadi added a comment -

          Patch looks good.
          +1

          Show
          Ravi Gummadi added a comment - Patch looks good. +1
          Hide
          Amareshwari Sriramadasu added a comment -

          Ran test-patch and ant test on my machine. All tests passed except the known failures (MAPREDUCE-1834 and MAPREDUCE-1925)

          Show
          Amareshwari Sriramadasu added a comment - Ran test-patch and ant test on my machine. All tests passed except the known failures ( MAPREDUCE-1834 and MAPREDUCE-1925 )
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch incorporates review comments.

          Show
          Amareshwari Sriramadasu added a comment - Patch incorporates review comments.
          Hide
          Ravi Gummadi added a comment -

          Some comments:
          (1) In DumpTypedBytes.java, LoadTypedBytes.java and in HadoopStreaming.java, in printUsage(), "$HADOOP_HOME/hadoop-streaming.jar" is mentioned, which is not the correct path of streaming jar. May be it is better to just say hadoop-streaming.jar instead of giving path.
          (2) In DumpTypedBytes.java, in "if (args.length == 0)", before printUsage() call, printing an error message like "too few arguments to dumptb" would be useful.
          (3) Would it be better to say in printUsage() in HadoopStreaming.java as "[streamjob]" to signify that streamjob option is optional ?
          (4) Description of options/arguments is missing in printUsage() in LoadTypedBytes and DumpTypedBytes. You can give the same description provided printUsage() of HadoopStreaming.java. For dumptb, should we say that the input can be either text file or sequence file ?

          Will let you know offline some very minor nits.

          Show
          Ravi Gummadi added a comment - Some comments: (1) In DumpTypedBytes.java, LoadTypedBytes.java and in HadoopStreaming.java, in printUsage(), "$HADOOP_HOME/hadoop-streaming.jar" is mentioned, which is not the correct path of streaming jar. May be it is better to just say hadoop-streaming.jar instead of giving path. (2) In DumpTypedBytes.java, in "if (args.length == 0)", before printUsage() call, printing an error message like "too few arguments to dumptb" would be useful. (3) Would it be better to say in printUsage() in HadoopStreaming.java as " [streamjob] " to signify that streamjob option is optional ? (4) Description of options/arguments is missing in printUsage() in LoadTypedBytes and DumpTypedBytes. You can give the same description provided printUsage() of HadoopStreaming.java. For dumptb, should we say that the input can be either text file or sequence file ? Will let you know offline some very minor nits.
          Hide
          Amareshwari Sriramadasu added a comment -

          Test failures are because of MAPREDUCE-1834 and MAPREDUCE-1925

          Show
          Amareshwari Sriramadasu added a comment - Test failures are because of MAPREDUCE-1834 and MAPREDUCE-1925
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12449235/patch-1911-1.txt
          against trunk revision 963986.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449235/patch-1911-1.txt against trunk revision 963986. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/298/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch adds usage message for streaming.jar also (MAPREDUCE-1916).

          Show
          Amareshwari Sriramadasu added a comment - Patch adds usage message for streaming.jar also ( MAPREDUCE-1916 ).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448951/patch-1911.txt
          against trunk revision 961542.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448951/patch-1911.txt against trunk revision 961542. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/292/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch fixing the above errors. Also formatted the help message to be displayed in 80 character lines.

          Show
          Amareshwari Sriramadasu added a comment - Patch fixing the above errors. Also formatted the help message to be displayed in 80 character lines.

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development