Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0
    • Component/s: examples
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For new to hadoop, try for getting help mesage for examples jobs provided in mapreduce. These Usage helps them in providing arguements.

      terasort job execution does not print Usage message instead throw exception.

      ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar terasort
      14/03/24 15:34:55 INFO terasort.TeraSort: starting
      java.lang.ArrayIndexOutOfBoundsException: 0
      at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:283)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
      at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
      at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

      1. 0002-MAPREDUCE-5807.patch
        13 kB
        Harsh J
      2. 0001-MAPREDUCE-5807.patch
        12 kB
        Rohith
      3. MAPREDUCE-5807.patch
        1 kB
        Rohith

        Issue Links

          Activity

          Hide
          Gera Shegalov added a comment -

          This looks great. Thanks Rohith] and Harsh J!

          Show
          Gera Shegalov added a comment - This looks great. Thanks Rohith ] and Harsh J !
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #128 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/128/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #128 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/128/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2069 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2069/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2069 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2069/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #871 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/871/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #871 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/871/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #137 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/137/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #137 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/137/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/136/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/136/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2086 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2086/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2086 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2086/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7357 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7357/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7357 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7357/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          Hide
          Harsh J added a comment -

          Thanks for the patch Rohith (and for the additional review Tsuyoshi Ozawa)!

          Committed to branch-2 and trunk.

          Show
          Harsh J added a comment - Thanks for the patch Rohith (and for the additional review Tsuyoshi Ozawa )! Committed to branch-2 and trunk.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12705321/0002-MAPREDUCE-5807.patch
          against trunk revision 3411732.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705321/0002-MAPREDUCE-5807.patch against trunk revision 3411732. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//console This message is automatically generated.
          Hide
          Tsuyoshi Ozawa added a comment -

          +1, LGTM too.

          Show
          Tsuyoshi Ozawa added a comment - +1, LGTM too.
          Hide
          Tsuyoshi Ozawa added a comment -

          +1, LGTM too.

          Show
          Tsuyoshi Ozawa added a comment - +1, LGTM too.
          Hide
          Rohith added a comment -

          Thanks for the updated patch. Changes looks good to me
          +1 (non-binding)

          Show
          Rohith added a comment - Thanks for the updated patch. Changes looks good to me +1 (non-binding)
          Hide
          Harsh J added a comment -

          Thanks Rohith! The changes appear right. I've refined your patch a bit to further constant-ise defaults used in the config-getters, and have corrected some wording. Here's now it now appears:

          Usage: terasort [-Dproperty=value] <in> <out>
          TeraSort configurations are:
          <mapreduce.terasort.num-rows>     Number of rows to generate during teragen.
          <mapreduce.terasort.num.partitions>     Number of partitions used for sampling.
          <mapreduce.terasort.partitions.sample>     Sample size for each partition.
          <mapreduce.terasort.final.sync>     Perform a disk-persisting hsync at end of every file-write.
          <mapreduce.terasort.use.terascheduler>     Use TeraScheduler for computing input split distribution.
          <mapreduce.terasort.simplepartitioner>     Use SimplePartitioner instead of TotalOrderPartitioner.
          <mapreduce.terasort.output.replication>     Replication factor to use for output data files.
          

          Let me know if the changes look good to you.

          +1 generally, will commit after jenkins run.

          Show
          Harsh J added a comment - Thanks Rohith! The changes appear right. I've refined your patch a bit to further constant-ise defaults used in the config-getters, and have corrected some wording. Here's now it now appears: Usage: terasort [-Dproperty=value] <in> <out> TeraSort configurations are: <mapreduce.terasort.num-rows> Number of rows to generate during teragen. <mapreduce.terasort.num.partitions> Number of partitions used for sampling. <mapreduce.terasort.partitions.sample> Sample size for each partition. <mapreduce.terasort. final .sync> Perform a disk-persisting hsync at end of every file-write. <mapreduce.terasort.use.terascheduler> Use TeraScheduler for computing input split distribution. <mapreduce.terasort.simplepartitioner> Use SimplePartitioner instead of TotalOrderPartitioner. <mapreduce.terasort.output.replication> Replication factor to use for output data files. Let me know if the changes look good to you. +1 generally, will commit after jenkins run.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12705039/0001-MAPREDUCE-5807.patch
          against trunk revision 7179f94.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705039/0001-MAPREDUCE-5807.patch against trunk revision 7179f94. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//console This message is automatically generated.
          Hide
          Rohith added a comment -

          Usage print as below along with configurations

          Usage: terasort <in> <out>
          TeraSort configurations are:
          <mapreduce.terasort.num-rows>     Number of rows to generate
          <mapreduce.terasort.num.partitions>     Number of partitions used for sampling
          <mapreduce.terasort.partitions.sample>     sampling size
          <mapreduce.terasort.final.sync>     Wheather to do final sync before the stream is closed
          <mapreduce.terasort.use.terascheduler>     Wheather to use tera scheduler
          <mapreduce.terasort.simplepartitioner>     Wheather to use simple partitioner
          <mapreduce.terasort.output.replication>     Number of replications to be stored for output data
          
          Show
          Rohith added a comment - Usage print as below along with configurations Usage: terasort <in> <out> TeraSort configurations are: <mapreduce.terasort.num-rows> Number of rows to generate <mapreduce.terasort.num.partitions> Number of partitions used for sampling <mapreduce.terasort.partitions.sample> sampling size <mapreduce.terasort.final.sync> Wheather to do final sync before the stream is closed <mapreduce.terasort.use.terascheduler> Wheather to use tera scheduler <mapreduce.terasort.simplepartitioner> Wheather to use simple partitioner <mapreduce.terasort.output.replication> Number of replications to be stored for output data
          Hide
          Rohith added a comment -

          Updated the patch, kindly review

          Show
          Rohith added a comment - Updated the patch, kindly review
          Hide
          Harsh J added a comment -

          Rohith - Would you be providing an updated patch on this? I'd also take it if it printed all optional config options in the regular -help itself, not to be too picky, as presently there's zero information.

          The enum approach mentioned by Gera sounds nice to maintain and use.

          Show
          Harsh J added a comment - Rohith - Would you be providing an updated patch on this? I'd also take it if it printed all optional config options in the regular -help itself, not to be too picky, as presently there's zero information. The enum approach mentioned by Gera sounds nice to maintain and use.
          Hide
          Gera Shegalov added a comment -

          Hi Rohith, documenting separately is an option. It just usually goes stale very quickly. If we could consolidate all conf keys into a single enum along with descriptions, and dump it on some "-help config", that would be very useful. That's my experience with TeraSort versions.

          Show
          Gera Shegalov added a comment - Hi Rohith , documenting separately is an option. It just usually goes stale very quickly. If we could consolidate all conf keys into a single enum along with descriptions, and dump it on some "-help config", that would be very useful. That's my experience with TeraSort versions.
          Hide
          Rohith added a comment -

          Gera Shegalov, It is good to document terasort configurations separately. I feel that command help containes only usage of command. Basically , user should get at least what are the args to be provided!!
          I see 7 configurations are spread across multiple classes. Adding all configurations in command line help, is it good?

          Show
          Rohith added a comment - Gera Shegalov , It is good to document terasort configurations separately. I feel that command help containes only usage of command. Basically , user should get at least what are the args to be provided!! I see 7 configurations are spread across multiple classes. Adding all configurations in command line help, is it good?
          Hide
          Gera Shegalov added a comment -

          Rohith, that's a good idea. Do you mind adding help for terasort-specific configuration keys mapreduce.terasort.*. They are spread out across multiple classes.

          Show
          Gera Shegalov added a comment - Rohith , that's a good idea. Do you mind adding help for terasort-specific configuration keys mapreduce.terasort.* . They are spread out across multiple classes.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12636320/MAPREDUCE-5807.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636320/MAPREDUCE-5807.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//console This message is automatically generated.
          Hide
          Rohith added a comment -

          Attached patch for printing usage.Pls review.

          Show
          Rohith added a comment - Attached patch for printing usage.Pls review.

            People

            • Assignee:
              Rohith
              Reporter:
              Rohith
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development