Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6682

TestMRCJCFileOutputCommitter fails intermittently

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None

      Description

      java.lang.AssertionError: Output directory not empty expected:<0> but was:<4>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.apache.hadoop.mapred.TestMRCJCFileOutputCommitter.testAbort(TestMRCJCFileOutputCommitter.java:153)
      

      PreCommit Report
      https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6434/testReport/

      1. MAPREDUCE-6682.04.patch
        7 kB
        Akira Ajisaka
      2. MAPREDUCE-6682.03.patch
        6 kB
        Akira Ajisaka
      3. MAPREDUCE-6682.02.patch
        5 kB
        Akira Ajisaka
      4. MAPREDUCE-6682.01.patch
        5 kB
        Akira Ajisaka
      5. MAPREDUCE-6682.00.patch
        5 kB
        Akira Ajisaka

        Activity

        Hide
        ajisakaa Akira Ajisaka added a comment -

        It looks like another test put files in the directory and then testAbort() asserted that there is no file in the directory. I reproduced this failure by the following steps:

        $ mvn test -Dtest=TestMRTimelineEventHandling#testMapreduceJobTimelineServiceEnabled
        $ mvn test -Dtest=TestMRCJCFileOutputCommitter#testAbort 
        

        We should remove outDir before testAbort() and cleanup temporary directories after TestMRTimelineEventHandling#testMapreduceJobTimelineServiceEnabled.

        Show
        ajisakaa Akira Ajisaka added a comment - It looks like another test put files in the directory and then testAbort() asserted that there is no file in the directory. I reproduced this failure by the following steps: $ mvn test -Dtest=TestMRTimelineEventHandling#testMapreduceJobTimelineServiceEnabled $ mvn test -Dtest=TestMRCJCFileOutputCommitter#testAbort We should remove outDir before testAbort() and cleanup temporary directories after TestMRTimelineEventHandling#testMapreduceJobTimelineServiceEnabled .
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Hi Haibo Chen, how is this issue going? If you don't have time to provide a patch, I'd like to take it over.

        Show
        ajisakaa Akira Ajisaka added a comment - Hi Haibo Chen , how is this issue going? If you don't have time to provide a patch, I'd like to take it over.
        Hide
        haibochen Haibo Chen added a comment -

        Hi Akira Ajisaka, sorry for not being able to get to this. Assigning this to you.

        Show
        haibochen Haibo Chen added a comment - Hi Akira Ajisaka , sorry for not being able to get to this. Assigning this to you.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Haibo Chen for reassigning. I'll provide a patch shortly.

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Haibo Chen for reassigning. I'll provide a patch shortly.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Attaching a patch to fix this issue.

        Show
        ajisakaa Akira Ajisaka added a comment - Attaching a patch to fix this issue.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        01 patch

        • Reuse localPathRoot, inDir, and outDir.
        • Removed unnecessarily null check for inDir and outDir.
        Show
        ajisakaa Akira Ajisaka added a comment - 01 patch Reuse localPathRoot , inDir , and outDir . Removed unnecessarily null check for inDir and outDir .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 28s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 6m 59s trunk passed
        +1 compile 0m 23s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 27s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 23s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 20s the patch passed
        +1 compile 0m 20s the patch passed
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 13s the patch passed
        +1 mvnsite 0m 24s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 30s the patch passed
        +1 javadoc 0m 13s the patch passed
        +1 unit 112m 51s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        126m 21s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810186/MAPREDUCE-6682.01.patch
        JIRA Issue MAPREDUCE-6682
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c223413dc5c7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e3ba9ad
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 59s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 23s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 30s the patch passed +1 javadoc 0m 13s the patch passed +1 unit 112m 51s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 126m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810186/MAPREDUCE-6682.01.patch JIRA Issue MAPREDUCE-6682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c223413dc5c7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e3ba9ad Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6554/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 39s trunk passed
        +1 compile 0m 25s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 31s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 24s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 25s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 26s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 9s the patch passed
        -1 unit 124m 26s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        137m 59s



        Reason Tests
        Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810183/MAPREDUCE-6682.00.patch
        JIRA Issue MAPREDUCE-6682
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bed5a5fcf8ad 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e3ba9ad
        Default Java 1.8.0_91
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 39s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 31s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 25s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 20 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 9s the patch passed -1 unit 124m 26s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 137m 59s Reason Tests Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810183/MAPREDUCE-6682.00.patch JIRA Issue MAPREDUCE-6682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bed5a5fcf8ad 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e3ba9ad Default Java 1.8.0_91 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6553/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        02 patch: Rebased.

        Show
        ajisakaa Akira Ajisaka added a comment - 02 patch: Rebased.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 32s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 25s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 26s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 23s the patch passed
        +1 javac 0m 23s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 36s the patch passed
        +1 javadoc 0m 12s the patch passed
        +1 unit 114m 59s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 26s The patch does not generate ASF License warnings.
        128m 29s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818272/MAPREDUCE-6682.02.patch
        JIRA Issue MAPREDUCE-6682
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b7769beb0723 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ea9f437
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6618/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6618/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 26s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 114m 59s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 128m 29s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818272/MAPREDUCE-6682.02.patch JIRA Issue MAPREDUCE-6682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b7769beb0723 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ea9f437 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6618/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6618/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Hi Brahma Reddy Battula, would you review this?

        Show
        ajisakaa Akira Ajisaka added a comment - Hi Brahma Reddy Battula , would you review this?
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Akira Ajisaka thanks for working on this issue and pinging me..
        Approach overall looks great.

        Some minor nits:
        1) TestMRCJCFileOutputCommitter: As testCommitter also calling FileUtil.fullyDelete(new File(outDir.toString())) at the end of testcase,how about adding in teardown..?
        2) TestMRTimelineEventHandling: do we need to handle inDir delete also..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Akira Ajisaka thanks for working on this issue and pinging me.. Approach overall looks great. Some minor nits: 1) TestMRCJCFileOutputCommitter : As testCommitter also calling FileUtil.fullyDelete(new File(outDir.toString())) at the end of testcase,how about adding in teardown..? 2) TestMRTimelineEventHandling : do we need to handle inDir delete also..?
        Hide
        ajisakaa Akira Ajisaka added a comment -

        TestMRCJCFileOutputCommitter: As testCommitter also calling FileUtil.fullyDelete(new File(outDir.toString())) at the end of testcase,how about adding in teardown..?

        Nice catch!

        TestMRTimelineEventHandling: do we need to handle inDir delete also..?

        Deleting outDir is sufficient to fix this test failure, but I'm thinking we should delete inDir as well to avoid possible test failure like this.

        I'll update the patch to add @After method in TestMRCJCFileOutputCommitter.

        Show
        ajisakaa Akira Ajisaka added a comment - TestMRCJCFileOutputCommitter: As testCommitter also calling FileUtil.fullyDelete(new File(outDir.toString())) at the end of testcase,how about adding in teardown..? Nice catch! TestMRTimelineEventHandling: do we need to handle inDir delete also..? Deleting outDir is sufficient to fix this test failure, but I'm thinking we should delete inDir as well to avoid possible test failure like this. I'll update the patch to add @After method in TestMRCJCFileOutputCommitter.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 6m 49s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 30s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 26s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 24s the patch passed
        +1 javac 0m 24s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 28s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 29s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 111m 19s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        123m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820385/MAPREDUCE-6682.03.patch
        JIRA Issue MAPREDUCE-6682
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 680550b52731 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 55d5993
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6644/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6644/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 6m 49s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 26s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 29s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 111m 19s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 123m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820385/MAPREDUCE-6682.03.patch JIRA Issue MAPREDUCE-6682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 680550b52731 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 55d5993 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6644/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6644/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Akira Ajisaka thanks for updating the patch..

        one minor nit:

        How about using specific outdir for this class using GenericTestUtils.getTestDir(String subdir) and clean in teardown()..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Akira Ajisaka thanks for updating the patch.. one minor nit: How about using specific outdir for this class using GenericTestUtils.getTestDir(String subdir) and clean in teardown()..?
        Hide
        ajisakaa Akira Ajisaka added a comment - - edited

        Thanks Brahma Reddy Battula. Updated the patch to use GenericTestUtils.

        Show
        ajisakaa Akira Ajisaka added a comment - - edited Thanks Brahma Reddy Battula . Updated the patch to use GenericTestUtils.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 3s trunk passed
        +1 compile 0m 25s trunk passed
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 31s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 24s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 27s the patch passed
        +1 javac 0m 27s the patch passed
        +1 checkstyle 0m 18s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11)
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 16s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 37s the patch passed
        +1 javadoc 0m 11s the patch passed
        +1 unit 110m 50s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        124m 7s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820632/MAPREDUCE-6682.04.patch
        JIRA Issue MAPREDUCE-6682
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9e4b8b4ade29 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8d06bda
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6646/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6646/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 3s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 31s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 27s the patch passed +1 javac 0m 27s the patch passed +1 checkstyle 0m 18s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: The patch generated 0 new + 10 unchanged - 1 fixed = 10 total (was 11) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 37s the patch passed +1 javadoc 0m 11s the patch passed +1 unit 110m 50s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 124m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820632/MAPREDUCE-6682.04.patch JIRA Issue MAPREDUCE-6682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9e4b8b4ade29 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8d06bda Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6646/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6646/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Akira Ajisaka thanks again for updating the patch... lgtm +1 after addressing the following..

        now FileUtil.fullyDelete(new File(outDir.toString())); might not require in testAbort()..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Akira Ajisaka thanks again for updating the patch... lgtm +1 after addressing the following.. now FileUtil.fullyDelete(new File(outDir.toString())); might not require in testAbort() ..?
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Probably FileUtil.fullyDelete is not required in testAbort() but I'd like to keep it for safety.
        If a test wrongly creates a file in outDir and testAbort() is executed first in TestMRCJCFileOutputCommitter, the test fails. This is because teardown() is not called if testAbort() is executed first.

        Show
        ajisakaa Akira Ajisaka added a comment - Probably FileUtil.fullyDelete is not required in testAbort() but I'd like to keep it for safety. If a test wrongly creates a file in outDir and testAbort() is executed first in TestMRCJCFileOutputCommitter, the test fails. This is because teardown() is not called if testAbort() is executed first.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Ok.. thanks for Clarification.. will commit today..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Ok.. thanks for Clarification.. will commit today..
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Committed to trunk.. Thanks Akira Ajisaka for your contribution.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Committed to trunk.. Thanks Akira Ajisaka for your contribution.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10210 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10210/)
        MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently (brahma: rev 8f1c374bec2451568f463ab68f7eb6db734ab14e)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRCJCFileOutputCommitter.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10210 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10210/ ) MAPREDUCE-6682 . TestMRCJCFileOutputCommitter fails intermittently (brahma: rev 8f1c374bec2451568f463ab68f7eb6db734ab14e) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRCJCFileOutputCommitter.java
        Hide
        ajisakaa Akira Ajisaka added a comment -
        Show
        ajisakaa Akira Ajisaka added a comment - Thank you Brahma Reddy Battula !
        Hide
        jlowe Jason Lowe added a comment -

        Thanks, Akira Ajisaka! I committed this to branch-2 and branch-2.8 as well.

        Show
        jlowe Jason Lowe added a comment - Thanks, Akira Ajisaka ! I committed this to branch-2 and branch-2.8 as well.

          People

          • Assignee:
            ajisakaa Akira Ajisaka
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development