Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0
    • Component/s: examples
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For new to hadoop, try for getting help mesage for examples jobs provided in mapreduce. These Usage helps them in providing arguements.

      terasort job execution does not print Usage message instead throw exception.

      ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar terasort
      14/03/24 15:34:55 INFO terasort.TeraSort: starting
      java.lang.ArrayIndexOutOfBoundsException: 0
      at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:283)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:325)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
      at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
      at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

      1. MAPREDUCE-5807.patch
        1 kB
        Rohith Sharma K S
      2. 0001-MAPREDUCE-5807.patch
        12 kB
        Rohith Sharma K S
      3. 0002-MAPREDUCE-5807.patch
        13 kB
        Harsh J

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          357d 18h 52m 1 Harsh J 17/Mar/15 05:28
          Open Open Patch Available Patch Available
          5h 4m 2 Rohith Sharma K S 17/Mar/15 10:27
          Patch Available Patch Available Resolved Resolved
          1d 1h 35m 1 Harsh J 18/Mar/15 12:02
          Hide
          Gera Shegalov added a comment -

          This looks great. Thanks Rohith Sharma K S] and Harsh J!

          Show
          Gera Shegalov added a comment - This looks great. Thanks Rohith Sharma K S ] and Harsh J !
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #128 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/128/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #128 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/128/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2069 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2069/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2069 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2069/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #871 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/871/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #871 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/871/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #137 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/137/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #137 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/137/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/136/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #136 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/136/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2086 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2086/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2086 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2086/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7357 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7357/)
          MAPREDUCE-5807. Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578)

          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java
          • hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7357 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7357/ ) MAPREDUCE-5807 . Print usage for TeraSort job. Contributed by Rohith. (harsh: rev 9d72f939759f407796ecb4715c2dc2f0d36d5578) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSortConfigKeys.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java
          Harsh J made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.8.0 [ 12329060 ]
          Resolution Fixed [ 1 ]
          Hide
          Harsh J added a comment -

          Thanks for the patch Rohith Sharma K S (and for the additional review Tsuyoshi Ozawa)!

          Committed to branch-2 and trunk.

          Show
          Harsh J added a comment - Thanks for the patch Rohith Sharma K S (and for the additional review Tsuyoshi Ozawa )! Committed to branch-2 and trunk.
          Harsh J made changes -
          Affects Version/s 2.6.0 [ 12327180 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12705321/0002-MAPREDUCE-5807.patch
          against trunk revision 3411732.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705321/0002-MAPREDUCE-5807.patch against trunk revision 3411732. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5305//console This message is automatically generated.
          Hide
          Tsuyoshi Ozawa added a comment -

          +1, LGTM too.

          Show
          Tsuyoshi Ozawa added a comment - +1, LGTM too.
          Hide
          Tsuyoshi Ozawa added a comment -

          +1, LGTM too.

          Show
          Tsuyoshi Ozawa added a comment - +1, LGTM too.
          Hide
          Rohith Sharma K S added a comment -

          Thanks for the updated patch. Changes looks good to me
          +1 (non-binding)

          Show
          Rohith Sharma K S added a comment - Thanks for the updated patch. Changes looks good to me +1 (non-binding)
          Harsh J made changes -
          Attachment 0002-MAPREDUCE-5807.patch [ 12705321 ]
          Hide
          Harsh J added a comment -

          Thanks Rohith! The changes appear right. I've refined your patch a bit to further constant-ise defaults used in the config-getters, and have corrected some wording. Here's now it now appears:

          Usage: terasort [-Dproperty=value] <in> <out>
          TeraSort configurations are:
          <mapreduce.terasort.num-rows>     Number of rows to generate during teragen.
          <mapreduce.terasort.num.partitions>     Number of partitions used for sampling.
          <mapreduce.terasort.partitions.sample>     Sample size for each partition.
          <mapreduce.terasort.final.sync>     Perform a disk-persisting hsync at end of every file-write.
          <mapreduce.terasort.use.terascheduler>     Use TeraScheduler for computing input split distribution.
          <mapreduce.terasort.simplepartitioner>     Use SimplePartitioner instead of TotalOrderPartitioner.
          <mapreduce.terasort.output.replication>     Replication factor to use for output data files.
          

          Let me know if the changes look good to you.

          +1 generally, will commit after jenkins run.

          Show
          Harsh J added a comment - Thanks Rohith! The changes appear right. I've refined your patch a bit to further constant-ise defaults used in the config-getters, and have corrected some wording. Here's now it now appears: Usage: terasort [-Dproperty=value] <in> <out> TeraSort configurations are: <mapreduce.terasort.num-rows> Number of rows to generate during teragen. <mapreduce.terasort.num.partitions> Number of partitions used for sampling. <mapreduce.terasort.partitions.sample> Sample size for each partition. <mapreduce.terasort. final .sync> Perform a disk-persisting hsync at end of every file-write. <mapreduce.terasort.use.terascheduler> Use TeraScheduler for computing input split distribution. <mapreduce.terasort.simplepartitioner> Use SimplePartitioner instead of TotalOrderPartitioner. <mapreduce.terasort.output.replication> Replication factor to use for output data files. Let me know if the changes look good to you. +1 generally, will commit after jenkins run.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12705039/0001-MAPREDUCE-5807.patch
          against trunk revision 7179f94.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705039/0001-MAPREDUCE-5807.patch against trunk revision 7179f94. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5301//console This message is automatically generated.
          Hide
          Rohith Sharma K S added a comment -

          Usage print as below along with configurations

          Usage: terasort <in> <out>
          TeraSort configurations are:
          <mapreduce.terasort.num-rows>     Number of rows to generate
          <mapreduce.terasort.num.partitions>     Number of partitions used for sampling
          <mapreduce.terasort.partitions.sample>     sampling size
          <mapreduce.terasort.final.sync>     Wheather to do final sync before the stream is closed
          <mapreduce.terasort.use.terascheduler>     Wheather to use tera scheduler
          <mapreduce.terasort.simplepartitioner>     Wheather to use simple partitioner
          <mapreduce.terasort.output.replication>     Number of replications to be stored for output data
          
          Show
          Rohith Sharma K S added a comment - Usage print as below along with configurations Usage: terasort <in> <out> TeraSort configurations are: <mapreduce.terasort.num-rows> Number of rows to generate <mapreduce.terasort.num.partitions> Number of partitions used for sampling <mapreduce.terasort.partitions.sample> sampling size <mapreduce.terasort.final.sync> Wheather to do final sync before the stream is closed <mapreduce.terasort.use.terascheduler> Wheather to use tera scheduler <mapreduce.terasort.simplepartitioner> Wheather to use simple partitioner <mapreduce.terasort.output.replication> Number of replications to be stored for output data
          Rohith Sharma K S made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Rohith Sharma K S added a comment -

          Updated the patch, kindly review

          Show
          Rohith Sharma K S added a comment - Updated the patch, kindly review
          Rohith Sharma K S made changes -
          Attachment 0001-MAPREDUCE-5807.patch [ 12705039 ]
          Harsh J made changes -
          Link This issue is duplicated by MAPREDUCE-1557 [ MAPREDUCE-1557 ]
          Harsh J made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Harsh J added a comment -

          Rohith Sharma K S - Would you be providing an updated patch on this? I'd also take it if it printed all optional config options in the regular -help itself, not to be too picky, as presently there's zero information.

          The enum approach mentioned by Gera sounds nice to maintain and use.

          Show
          Harsh J added a comment - Rohith Sharma K S - Would you be providing an updated patch on this? I'd also take it if it printed all optional config options in the regular -help itself, not to be too picky, as presently there's zero information. The enum approach mentioned by Gera sounds nice to maintain and use.
          Hide
          Gera Shegalov added a comment -

          Hi Rohith Sharma K S, documenting separately is an option. It just usually goes stale very quickly. If we could consolidate all conf keys into a single enum along with descriptions, and dump it on some "-help config", that would be very useful. That's my experience with TeraSort versions.

          Show
          Gera Shegalov added a comment - Hi Rohith Sharma K S , documenting separately is an option. It just usually goes stale very quickly. If we could consolidate all conf keys into a single enum along with descriptions, and dump it on some "-help config", that would be very useful. That's my experience with TeraSort versions.
          Hide
          Rohith Sharma K S added a comment -

          Gera Shegalov, It is good to document terasort configurations separately. I feel that command help containes only usage of command. Basically , user should get at least what are the args to be provided!!
          I see 7 configurations are spread across multiple classes. Adding all configurations in command line help, is it good?

          Show
          Rohith Sharma K S added a comment - Gera Shegalov , It is good to document terasort configurations separately. I feel that command help containes only usage of command. Basically , user should get at least what are the args to be provided!! I see 7 configurations are spread across multiple classes. Adding all configurations in command line help, is it good?
          Hide
          Gera Shegalov added a comment -

          Rohith Sharma K S, that's a good idea. Do you mind adding help for terasort-specific configuration keys mapreduce.terasort.*. They are spread out across multiple classes.

          Show
          Gera Shegalov added a comment - Rohith Sharma K S , that's a good idea. Do you mind adding help for terasort-specific configuration keys mapreduce.terasort.* . They are spread out across multiple classes.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12636320/MAPREDUCE-5807.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636320/MAPREDUCE-5807.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4458//console This message is automatically generated.
          Rohith Sharma K S made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Rohith Sharma K S made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-5807.patch [ 12636320 ]
          Hide
          Rohith Sharma K S added a comment -

          Attached patch for printing usage.Pls review.

          Show
          Rohith Sharma K S added a comment - Attached patch for printing usage.Pls review.
          Rohith Sharma K S created issue -

            People

            • Assignee:
              Rohith Sharma K S
              Reporter:
              Rohith Sharma K S
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development