Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2398

MRBench: setting the baseDir parameter has no effect

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: benchmarks
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The optional -baseDir parameter lets user specify the base DFS path for output/input of MRBench.

      However, the two private variables INPUT_DIR and OUTPUT_DIR (MRBench.java) are not updated in the case that the default value of -baseDir is actually overwritten by the user. Hence any input and output is always written to the default locations (/benchmarks/MRBench/...), even though the user-supplied location for -baseDir is created (and eventually deleted again) on HDFS.

      The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of March 21, 2011.

      1. MAPREDUCE-2398_0.20.2.patch
        0.7 kB
        Michael Noll
      2. MAPREDUCE-2398_v2-0.20.203.0.patch
        2 kB
        Michael Noll
      3. MAPREDUCE-2398_v2-trunk.patch
        2 kB
        Michael Noll
      4. MAPREDUCE-2398.2.patch
        3 kB
        Wilfred Spiegelenburg
      5. MAPREDUCE-2398-trunk.patch
        0.7 kB
        Michael Noll

        Activity

        Hide
        miguno Michael Noll added a comment -

        Patch in unified diff format (old/src/... versus new/src/...) for Hadoop 0.20.2 attached.

        Show
        miguno Michael Noll added a comment - Patch in unified diff format (old/src/... versus new/src/...) for Hadoop 0.20.2 attached.
        Hide
        miguno Michael Noll added a comment -

        Patch in svn diff format for trunk.

        Show
        miguno Michael Noll added a comment - Patch in svn diff format for trunk.
        Hide
        miguno Michael Noll added a comment -

        Just let me know if everything regarding the patch is ok (patch itself + JIRA ticket info).

        Show
        miguno Michael Noll added a comment - Just let me know if everything regarding the patch is ok (patch itself + JIRA ticket info).
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12474163/MAPREDUCE-2398-trunk.patch
        against trunk revision 1082703.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//testReport/
        Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12474163/MAPREDUCE-2398-trunk.patch against trunk revision 1082703. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/142//console This message is automatically generated.
        Hide
        miguno Michael Noll added a comment -

        Hmm. From what I see in the Hudson test results, the two failed tests (org.apache.hadoop.mapred.TestFairSchedulerSystem.testFairSchedulerSystem and org.apache.hadoop.raid.TestBlockFixer.testMaxPendingFiles) have nothing to do with this patch. Or am I missing something?

        On a related note, what is the current approach/policy to write proper unit tests for benchmark classes like MRBench? In the example of this ticket, how should the unit test perform checking whether MRBench picks up the -baseDir parameter properly from the command line? By checking variable assignments (= events inside the code), or by checking the directory actually created and subsequently used on HDFS (= events outside the code)?

        Show
        miguno Michael Noll added a comment - Hmm. From what I see in the Hudson test results , the two failed tests ( org.apache.hadoop.mapred.TestFairSchedulerSystem.testFairSchedulerSystem and org.apache.hadoop.raid.TestBlockFixer.testMaxPendingFiles ) have nothing to do with this patch. Or am I missing something? On a related note, what is the current approach/policy to write proper unit tests for benchmark classes like MRBench? In the example of this ticket, how should the unit test perform checking whether MRBench picks up the -baseDir parameter properly from the command line? By checking variable assignments (= events inside the code), or by checking the directory actually created and subsequently used on HDFS (= events outside the code)?
        Hide
        qwertymaniac Harsh J added a comment -

        Michael,

        Thanks for the patch!

        One minor nit: Can "mr_input" and "mr_output" strings be constant-ized and reused across both setting statements?

        For the test question, I think events outside code approach is better.

        Show
        qwertymaniac Harsh J added a comment - Michael, Thanks for the patch! One minor nit: Can "mr_input" and "mr_output" strings be constant-ized and reused across both setting statements? For the test question, I think events outside code approach is better.
        Hide
        miguno Michael Noll added a comment -

        New version of patch, integrating Harsh J's suggestions. This patch is based on Hadoop release 0.20.203.0.

        Show
        miguno Michael Noll added a comment - New version of patch, integrating Harsh J's suggestions. This patch is based on Hadoop release 0.20.203.0.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12493159/MAPREDUCE-2398_v2-0.20.203.0.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/615//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493159/MAPREDUCE-2398_v2-0.20.203.0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/615//console This message is automatically generated.
        Hide
        miguno Michael Noll added a comment -

        New version of patch, integrating Harsh J's suggestions. This patch is based on trunk (SVN rev 1166009 / git 2647680).

        Show
        miguno Michael Noll added a comment - New version of patch, integrating Harsh J's suggestions. This patch is based on trunk (SVN rev 1166009 / git 2647680).
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12493285/MAPREDUCE-2398_v2-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:

        org.apache.hadoop.yarn.server.nodemanager.TestNMAuditLogger

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493285/MAPREDUCE-2398_v2-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.TestNMAuditLogger +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/621//console This message is automatically generated.
        Hide
        miguno Michael Noll added a comment -

        From what I can tell the error

        -1 core tests. The patch failed these unit tests:
        
        org.apache.hadoop.yarn.server.nodemanager.TestNMAuditLogger
        

        is not related to this patch.

        Show
        miguno Michael Noll added a comment - From what I can tell the error -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.TestNMAuditLogger is not related to this patch.
        Hide
        qwertymaniac Harsh J added a comment -

        +1 Both of these patches lgtm.

        Show
        qwertymaniac Harsh J added a comment - +1 Both of these patches lgtm.
        Hide
        yanghaogn Yang Hao added a comment -

        This problem still exists in the version Hadoop-2.3.0, so please fix it in the java "org.apache.hadoop.mapred.MRBench".Sometimes we don't have the permission to operate all the HDFS file, so we have to change the BASE_DIR , and the INPUT_DIR and OUTPUT_DIR should be updated when BASE_DIR is set. Only two extra lines be put into the java source code can solve the bug

        Show
        yanghaogn Yang Hao added a comment - This problem still exists in the version Hadoop-2.3.0, so please fix it in the java "org.apache.hadoop.mapred.MRBench".Sometimes we don't have the permission to operate all the HDFS file, so we have to change the BASE_DIR , and the INPUT_DIR and OUTPUT_DIR should be updated when BASE_DIR is set. Only two extra lines be put into the java source code can solve the bug
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Moving bugs out of previously closed releases into the next minor release 2.8.0.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Moving bugs out of previously closed releases into the next minor release 2.8.0.
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        This seems to have gone stale for a long time and just ran into this.

        I have updated the patch and made sure it works passing in the value from the command line. As part of the change I also cleaned up the directory creation and cleanup.
        The INPUT_DIR automatically gets created by the generateTextFile() call. The clean up should not delete the BASE_DIR because it could exist and have other data in it. Only the OUTPUT_DIR and INPUT_DIR that were created for the run should be removed for cleanup.

        Show
        wilfreds Wilfred Spiegelenburg added a comment - This seems to have gone stale for a long time and just ran into this. I have updated the patch and made sure it works passing in the value from the command line. As part of the change I also cleaned up the directory creation and cleanup. The INPUT_DIR automatically gets created by the generateTextFile() call. The clean up should not delete the BASE_DIR because it could exist and have other data in it. Only the OUTPUT_DIR and INPUT_DIR that were created for the run should be removed for cleanup.
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        Yang Hao do you mind if I assign this one to me?

        Show
        wilfreds Wilfred Spiegelenburg added a comment - Yang Hao do you mind if I assign this one to me?
        Hide
        qwertymaniac Harsh J added a comment -

        (Trying a resolve to reset the IN-PROGRESS state as there seems to be no option to reset it after recent changes?; Will reopen shortly)

        Show
        qwertymaniac Harsh J added a comment - (Trying a resolve to reset the IN-PROGRESS state as there seems to be no option to reset it after recent changes?; Will reopen shortly)
        Hide
        qwertymaniac Harsh J added a comment -

        (Sorry for all the extra status noise; Marking as Patch-Available finally)

        Show
        qwertymaniac Harsh J added a comment - (Sorry for all the extra status noise; Marking as Patch-Available finally)
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 33s trunk passed
        +1 compile 0m 18s trunk passed with JDK v1.8.0_77
        +1 compile 0m 22s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 12s trunk passed
        +1 mvnsite 0m 28s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 26s trunk passed
        +1 javadoc 0m 10s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 21s the patch passed
        +1 compile 0m 17s the patch passed with JDK v1.8.0_77
        +1 javac 0m 17s the patch passed
        +1 compile 0m 20s the patch passed with JDK v1.7.0_95
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 35s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95
        -1 unit 101m 13s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_77.
        -1 unit 105m 18s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 26s Patch does not generate ASF License warnings.
        219m 46s



        Reason Tests
        JDK v1.8.0_77 Failed junit tests hadoop.mapreduce.v2.TestUberAM
          hadoop.mapred.TestMRCJCFileOutputCommitter
        JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.TestUberAM
          hadoop.mapred.TestMRCJCFileOutputCommitter



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799893/MAPREDUCE-2398.2.patch
        JIRA Issue MAPREDUCE-2398
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4c3c94816e62 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1e48eef
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_77.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 33s trunk passed +1 compile 0m 18s trunk passed with JDK v1.8.0_77 +1 compile 0m 22s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 26s trunk passed +1 javadoc 0m 10s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 21s the patch passed +1 compile 0m 17s the patch passed with JDK v1.8.0_77 +1 javac 0m 17s the patch passed +1 compile 0m 20s the patch passed with JDK v1.7.0_95 +1 javac 0m 20s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 35s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 11s the patch passed with JDK v1.7.0_95 -1 unit 101m 13s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_77. -1 unit 105m 18s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 219m 46s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.mapreduce.v2.TestUberAM   hadoop.mapred.TestMRCJCFileOutputCommitter JDK v1.7.0_95 Failed junit tests hadoop.mapreduce.v2.TestUberAM   hadoop.mapred.TestMRCJCFileOutputCommitter Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799893/MAPREDUCE-2398.2.patch JIRA Issue MAPREDUCE-2398 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4c3c94816e62 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1e48eef Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6453/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        test failures are not related to this patch:

        • TestMRCJCFileOutputCommitter has been failing in multiple builds
        • TestUberAM passes for me locally
        Show
        wilfreds Wilfred Spiegelenburg added a comment - test failures are not related to this patch: TestMRCJCFileOutputCommitter has been failing in multiple builds TestUberAM passes for me locally
        Hide
        qwertymaniac Harsh J added a comment -

        +1, committing shortly.

        Show
        qwertymaniac Harsh J added a comment - +1, committing shortly.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9643 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9643/)
        MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. (harsh: rev 7da5847cf10bc61f2131d578f16ee37cb32c4639)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9643 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9643/ ) MAPREDUCE-2398 . MRBench: setting the baseDir parameter has no effect. (harsh: rev 7da5847cf10bc61f2131d578f16ee37cb32c4639) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/MRBench.java
        Hide
        wilfreds Wilfred Spiegelenburg added a comment -

        Harsh J thank you for the quick review and commit

        Show
        wilfreds Wilfred Spiegelenburg added a comment - Harsh J thank you for the quick review and commit

          People

          • Assignee:
            wilfreds Wilfred Spiegelenburg
            Reporter:
            miguno Michael Noll
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development