Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5436

Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In YARN-2264, a race in DrainDispatcher was fixed. Unfortunately, it also exists in AsyncDispatcher (this was found and ignored in YARN-3878 but never documented...). In YARN-2991, another DrainDispatcher bug was fixed by letting DrainDispatcher reuse some AsyncDispatcher method because AsyncDispatcher doesn't have such issue. However, this shadows YARN-2264, and now similar race reappears in Tez unit tests (probably also YARN unit tests also).

      1. YARN-5436.4.patch
        4 kB
        Zhiyuan Yang
      2. YARN-5436.3.patch
        3 kB
        Zhiyuan Yang
      3. YARN-5436.2.patch
        3 kB
        Zhiyuan Yang
      4. YARN-5436.1.patch
        4 kB
        Zhiyuan Yang

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10175 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10175/)
          YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (gtcarrera9: rev 7086fc72eebc41fd174d91839ed703c014aac920)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/DrainDispatcher.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10175 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10175/ ) YARN-5436 . Race in AsyncDispatcher can cause random test failures in Tez (gtcarrera9: rev 7086fc72eebc41fd174d91839ed703c014aac920) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/DrainDispatcher.java
          Hide
          gtCarrera9 Li Lu added a comment -

          Committed to trunk and branch-2. Thanks Zhiyuan Yang for the work and Rohith Sharma K S for the review!

          Show
          gtCarrera9 Li Lu added a comment - Committed to trunk and branch-2. Thanks Zhiyuan Yang for the work and Rohith Sharma K S for the review!
          Hide
          gtCarrera9 Li Lu added a comment -

          Will commit this patch shortly.

          Show
          gtCarrera9 Li Lu added a comment - Will commit this patch shortly.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks for the clarification, especially race can still happen without invoking dispatcher.serviceStop() is main reason for test failures. This would solve YARN test failures also.

          +1 LGTM

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks for the clarification, especially race can still happen without invoking dispatcher.serviceStop() is main reason for test failures. This would solve YARN test failures also. +1 LGTM
          Hide
          aplusplus Zhiyuan Yang added a comment -

          Rohith Sharma K S Thanks for reviewing! You are right in the sense this patch is mostly letting DrainDispatcher not reuse AsyncDispatcher's drained field, but the fix for YARN-2991 is still there.

          does small tiny race is causing TEZ test failures?

          Yes. In Tez UT tests, invocation of dispatcher.await() finished without handling all events and assertion after dispatcher.await() failed. This race condition only happens when queue is almost empty, which is exactly the case in Tez UT tests.

          If so would it be good to fix in AsyncDispatcher rather adding full duplicate code.

          The root cause of race is we cannot guarantee we enqueue event and update drained atomically. I didn't find a way to fix this without adding more synchronization which is a very expensive fix for a minimum benefit. YARN-3878 discussed about this race and decided to ignore it for the same reason.

          How about adding additional check before adding into event queue to avoid a race?

          While this may avoid enqueuing last event, race can still happen without invoking dispatcher.serviceStop(). Actually in Tez UT test, we never invoke dispatcher.serviceStop().

          Show
          aplusplus Zhiyuan Yang added a comment - Rohith Sharma K S Thanks for reviewing! You are right in the sense this patch is mostly letting DrainDispatcher not reuse AsyncDispatcher's drained field, but the fix for YARN-2991 is still there. does small tiny race is causing TEZ test failures? Yes. In Tez UT tests, invocation of dispatcher.await() finished without handling all events and assertion after dispatcher.await() failed. This race condition only happens when queue is almost empty, which is exactly the case in Tez UT tests. If so would it be good to fix in AsyncDispatcher rather adding full duplicate code. The root cause of race is we cannot guarantee we enqueue event and update drained atomically. I didn't find a way to fix this without adding more synchronization which is a very expensive fix for a minimum benefit. YARN-3878 discussed about this race and decided to ignore it for the same reason. How about adding additional check before adding into event queue to avoid a race? While this may avoid enqueuing last event, race can still happen without invoking dispatcher.serviceStop(). Actually in Tez UT test, we never invoke dispatcher.serviceStop().
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Zhiyuan for providing patch! Basically I see that patch is reverting YARN-2991.

          Couples of doubts, does small tiny race is causing TEZ test failures? If so would it be good to fix in AsyncDispatcher rather adding full duplicate code.
          How about adding additional check before adding into event queue to avoid a race?

          diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
          index f5361c8..a162690 100644
          --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
          +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
          @@ -247,6 +247,12 @@ public void handle(Event event) {
                   LOG.warn("Very low remaining capacity in the event-queue: "
                       + remCapacity);
                 }
          +
          +      if (blockNewEvents) {
          +        drained = eventQueue.isEmpty();
          +        return;
          +      }
          +
                 try {
                   eventQueue.put(event);
                 } catch (InterruptedException e) {
          
          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Zhiyuan for providing patch! Basically I see that patch is reverting YARN-2991 . Couples of doubts, does small tiny race is causing TEZ test failures? If so would it be good to fix in AsyncDispatcher rather adding full duplicate code. How about adding additional check before adding into event queue to avoid a race? diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java index f5361c8..a162690 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java @@ -247,6 +247,12 @@ public void handle(Event event) { LOG.warn( "Very low remaining capacity in the event-queue: " + remCapacity); } + + if (blockNewEvents) { + drained = eventQueue.isEmpty(); + return ; + } + try { eventQueue.put(event); } catch (InterruptedException e) {
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 38s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 55s trunk passed
          +1 javadoc 0m 27s trunk passed
          +1 mvninstall 0m 26s the patch passed
          +1 compile 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          +1 checkstyle 0m 16s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 4s the patch passed
          +1 javadoc 0m 29s the patch passed
          +1 unit 2m 19s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          16m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820582/YARN-5436.4.patch
          JIRA Issue YARN-5436
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1241e8c12390 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb7ff0c
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12531/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12531/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 38s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 55s trunk passed +1 javadoc 0m 27s trunk passed +1 mvninstall 0m 26s the patch passed +1 compile 0m 24s the patch passed +1 javac 0m 24s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 4s the patch passed +1 javadoc 0m 29s the patch passed +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 16m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820582/YARN-5436.4.patch JIRA Issue YARN-5436 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1241e8c12390 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb7ff0c Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12531/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12531/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          Fix LGTM, +1. I'll wait for 24 hrs for more comments.

          Show
          gtCarrera9 Li Lu added a comment - Fix LGTM, +1. I'll wait for 24 hrs for more comments.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 47s trunk passed
          +1 compile 0m 32s trunk passed
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 37s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 6s trunk passed
          +1 javadoc 0m 32s trunk passed
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 28s the patch passed
          +1 javac 0m 28s the patch passed
          +1 checkstyle 0m 18s the patch passed
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 15s the patch passed
          +1 javadoc 0m 30s the patch passed
          +1 unit 2m 36s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          19m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820572/YARN-5436.3.patch
          JIRA Issue YARN-5436
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f1b6e06f22a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb7ff0c
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12529/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12529/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 47s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 32s trunk passed +1 mvninstall 0m 29s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 15s the patch passed +1 javadoc 0m 30s the patch passed +1 unit 2m 36s hadoop-yarn-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 19m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820572/YARN-5436.3.patch JIRA Issue YARN-5436 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f1b6e06f22a9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb7ff0c Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12529/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12529/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aplusplus Zhiyuan Yang added a comment -

          Uploaded the patch that doesn't use java 8 feature for branch-2 sake.

          Show
          aplusplus Zhiyuan Yang added a comment - Uploaded the patch that doesn't use java 8 feature for branch-2 sake.
          Hide
          aplusplus Zhiyuan Yang added a comment - - edited

          Thanks Li Lu for reviewing the patch. Sorry for misusing the term 'data race'. Already rephrased the comments.

          Show
          aplusplus Zhiyuan Yang added a comment - - edited Thanks Li Lu for reviewing the patch. Sorry for misusing the term 'data race'. Already rephrased the comments.
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks for the work Zhiyuan Yang! The FIXME part in AsyncDispatcher appears to be confusing: There is no data race (per Java memory model's definition) with the volatile variable drained. Maybe you'd like to rephrase a little bit to express the potential nondeterminism?

          Other changes in DrainedDispatcher appears to be fine to me.

          Show
          gtCarrera9 Li Lu added a comment - Thanks for the work Zhiyuan Yang ! The FIXME part in AsyncDispatcher appears to be confusing: There is no data race (per Java memory model's definition) with the volatile variable drained . Maybe you'd like to rephrase a little bit to express the potential nondeterminism? Other changes in DrainedDispatcher appears to be fine to me.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 48s trunk passed
          +1 compile 0m 27s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 33s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 0m 56s trunk passed
          +1 javadoc 0m 27s trunk passed
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 26s the patch passed
          +1 javac 0m 26s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 5s the patch passed
          +1 javadoc 0m 24s the patch passed
          +1 unit 2m 18s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          17m 26s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820553/YARN-5436.2.patch
          JIRA Issue YARN-5436
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b5dd8fb851ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54fe17a
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12528/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12528/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 48s trunk passed +1 compile 0m 27s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 33s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 27s trunk passed +1 mvninstall 0m 29s the patch passed +1 compile 0m 26s the patch passed +1 javac 0m 26s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 5s the patch passed +1 javadoc 0m 24s the patch passed +1 unit 2m 18s hadoop-yarn-common in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 17m 26s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820553/YARN-5436.2.patch JIRA Issue YARN-5436 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b5dd8fb851ba 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12528/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12528/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 12s trunk passed
          +1 compile 0m 33s trunk passed
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 6s trunk passed
          +1 javadoc 0m 31s trunk passed
          +1 mvninstall 0m 31s the patch passed
          +1 compile 0m 30s the patch passed
          -1 javac 0m 30s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 7s the patch passed
          +1 javadoc 0m 25s the patch passed
          +1 unit 2m 15s hadoop-yarn-common in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          18m 24s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820544/YARN-5436.1.patch
          JIRA Issue YARN-5436
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2325af36e225 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 54fe17a
          Default Java 1.8.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-YARN-Build/12526/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12526/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12526/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12526/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 12s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 31s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 30s the patch passed -1 javac 0m 30s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: The patch generated 2 new + 7 unchanged - 0 fixed = 9 total (was 7) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 7s the patch passed +1 javadoc 0m 25s the patch passed +1 unit 2m 15s hadoop-yarn-common in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 18m 24s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820544/YARN-5436.1.patch JIRA Issue YARN-5436 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2325af36e225 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 54fe17a Default Java 1.8.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-YARN-Build/12526/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12526/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12526/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/12526/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aplusplus Zhiyuan Yang added a comment -

          Race in AsyncDispatcher has been found and ignored in YARN-3887. Leave it there for now.

          Show
          aplusplus Zhiyuan Yang added a comment - Race in AsyncDispatcher has been found and ignored in YARN-3887 . Leave it there for now.
          Hide
          aplusplus Zhiyuan Yang added a comment -

          Upload the patch that fixes problems only in DrainDispatcher and documents minor race condition in AsyncDispatcher. Please help review. Jian He, Rohith Sharma K S, Varun Saxena.

          Show
          aplusplus Zhiyuan Yang added a comment - Upload the patch that fixes problems only in DrainDispatcher and documents minor race condition in AsyncDispatcher. Please help review. Jian He , Rohith Sharma K S , Varun Saxena .
          Hide
          aplusplus Zhiyuan Yang added a comment -

          Data race can cause RM stop without handling last enqueued event.

          Show
          aplusplus Zhiyuan Yang added a comment - Data race can cause RM stop without handling last enqueued event.

            People

            • Assignee:
              aplusplus Zhiyuan Yang
              Reporter:
              aplusplus Zhiyuan Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development