Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6731

TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling() may fail for concurrent tests

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling() uses the default file-system storage directory, and is brittle against concurrent tests.

      We should use a unique storage directory for the tests.

      1. MAPREDUCE-6731.01.patch
        7 kB
        Sangjin Lee
      2. MAPREDUCE-6731.02.patch
        7 kB
        Sangjin Lee

        Issue Links

          Activity

          Hide
          sjlee0 Sangjin Lee added a comment -

          Posted patch v.1.

          This uses a test directory local to the build, instead of the global shared storage directory. I also made the path separator cross-platform.

          Show
          sjlee0 Sangjin Lee added a comment - Posted patch v.1. This uses a test directory local to the build, instead of the global shared storage directory. I also made the path separator cross-platform.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 15s trunk passed
          +1 compile 0m 23s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 0m 24s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 22s the patch passed
          +1 compile 0m 20s the patch passed
          +1 javac 0m 20s the patch passed
          +1 checkstyle 0m 13s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 28s the patch passed
          +1 javadoc 0m 10s the patch passed
          -1 unit 119m 38s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          132m 38s



          Reason Tests
          Failed junit tests hadoop.mapred.TestMRCJCFileOutputCommitter



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817219/MAPREDUCE-6731.01.patch
          JIRA Issue MAPREDUCE-6731
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ab36a18eddd1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0fd3980
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 15s trunk passed +1 compile 0m 23s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 28s the patch passed +1 javadoc 0m 10s the patch passed -1 unit 119m 38s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 132m 38s Reason Tests Failed junit tests hadoop.mapred.TestMRCJCFileOutputCommitter Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817219/MAPREDUCE-6731.01.patch JIRA Issue MAPREDUCE-6731 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ab36a18eddd1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0fd3980 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6606/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          sjlee0 Sangjin Lee added a comment -

          The test failure is unrelated. I would greatly appreciate your review. Thanks!

          Show
          sjlee0 Sangjin Lee added a comment - The test failure is unrelated. I would greatly appreciate your review. Thanks!
          Hide
          vrushalic Vrushali C added a comment -

          +1
          Thanks Sangjin Lee for the patch, looks good to me.

          Show
          vrushalic Vrushali C added a comment - +1 Thanks Sangjin Lee for the patch, looks good to me.
          Hide
          sjlee0 Sangjin Lee added a comment -

          Posted patch v.2. A small change in the directory name (to make it shorter).

          Show
          sjlee0 Sangjin Lee added a comment - Posted patch v.2. A small change in the directory name (to make it shorter).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 0s trunk passed
          +1 compile 0m 24s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 29s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          +1 findbugs 0m 24s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 22s the patch passed
          +1 javac 0m 22s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 28s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 31s the patch passed
          +1 javadoc 0m 10s the patch passed
          -1 unit 119m 17s hadoop-mapreduce-client-jobclient in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          131m 56s



          Reason Tests
          Failed junit tests hadoop.mapred.TestClusterMapReduceTestCase
          Timed out junit tests org.apache.hadoop.mapred.TestMRIntermediateDataEncryption



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817277/MAPREDUCE-6731.02.patch
          JIRA Issue MAPREDUCE-6731
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d242c81e02ae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c447efe
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 31s the patch passed +1 javadoc 0m 10s the patch passed -1 unit 119m 17s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 131m 56s Reason Tests Failed junit tests hadoop.mapred.TestClusterMapReduceTestCase Timed out junit tests org.apache.hadoop.mapred.TestMRIntermediateDataEncryption Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817277/MAPREDUCE-6731.02.patch JIRA Issue MAPREDUCE-6731 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d242c81e02ae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c447efe Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6607/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          varun_saxena Varun Saxena added a comment -

          +1. Test failures look unrelated and pass for me in local.
          Will commit it later today unless someone has any further comments.

          Show
          varun_saxena Varun Saxena added a comment - +1. Test failures look unrelated and pass for me in local. Will commit it later today unless someone has any further comments.
          Hide
          varun_saxena Varun Saxena added a comment -

          Committed to trunk.
          Thanks Sangjin Lee for your contribution and Vrushali C for the review.

          Show
          varun_saxena Varun Saxena added a comment - Committed to trunk. Thanks Sangjin Lee for your contribution and Vrushali C for the review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10079 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10079/)
          MAPREDUCE-6731. (varunsaxena: rev d14e729a55c0ca4053c537290392b74f732ce939)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10079 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10079/ ) MAPREDUCE-6731 . (varunsaxena: rev d14e729a55c0ca4053c537290392b74f732ce939) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java

            People

            • Assignee:
              sjlee0 Sangjin Lee
              Reporter:
              sjlee0 Sangjin Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development