Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3990

MRBench allows Long-sized input-lines value but parses CLI argument as an Integer

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: benchmarks
    • Labels:

      Description

      MRBench has the following method:

      public void generateTextFile(FileSystem fs, Path inputFile, long numLines, Order sortOrder) { ... }
      

      The method is already set to accept a long datatype for numLines, for generating very large amount of data.

      However, in MRBench#run(...), the inputLines CLI parameter is parsed via an Integer.parseInt, causing numbers passed > Integer.MAX_VALUE to throw NumberFormatExceptions as a result.

      The parsing should be Long.parseLong and the inputLines datatype should be switched to the same type as passed to the method (long).

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12517679/MAPREDUCE-3990.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2022//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2022//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517679/MAPREDUCE-3990.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2022//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2022//console This message is automatically generated.
        Hide
        qwertymaniac Harsh J added a comment -

        Any comments on the patch? It is a trivial parsing bug-fix, but I'd like to wait for a +1 in any case.

        Show
        qwertymaniac Harsh J added a comment - Any comments on the patch? It is a trivial parsing bug-fix, but I'd like to wait for a +1 in any case.
        Hide
        qwertymaniac Harsh J added a comment -

        Given this patch's trivialness and that it merely enhances the tool rather than do anything negative, and that it has passed the QA run just fine, I'll commit it in by monday EOD unless anyone has any objections.

        Thanks,
        Harsh

        Show
        qwertymaniac Harsh J added a comment - Given this patch's trivialness and that it merely enhances the tool rather than do anything negative, and that it has passed the QA run just fine, I'll commit it in by monday EOD unless anyone has any objections. Thanks, Harsh
        Hide
        qwertymaniac Harsh J added a comment -

        Committed revision 1343247 to trunk.

        Show
        qwertymaniac Harsh J added a comment - Committed revision 1343247 to trunk.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2364 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2364/)
        MAPREDUCE-3990. MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2364 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2364/ ) MAPREDUCE-3990 . MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2291 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2291/)
        MAPREDUCE-3990. MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2291 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2291/ ) MAPREDUCE-3990 . MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2310 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2310/)
        MAPREDUCE-3990. MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2310 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2310/ ) MAPREDUCE-3990 . MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1060 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1060/)
        MAPREDUCE-3990. MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1060 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1060/ ) MAPREDUCE-3990 . MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1094 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1094/)
        MAPREDUCE-3990. MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1094 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1094/ ) MAPREDUCE-3990 . MRBench allows Long-sized input-lines value but parses CLI argument as an Integer. (harsh) (Revision 1343247) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343247 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java

          People

          • Assignee:
            qwertymaniac Harsh J
            Reporter:
            qwertymaniac Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development