Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6678

Handle IllegalStateException in Async Scheduling mode of CapacityScheduler

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0, 3.0.0-alpha3
    • Fix Version/s: 2.9.0, 3.0.0-beta1
    • Component/s: capacityscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Error log:

      java.lang.IllegalStateException: Trying to reserve container container_e10_1495599791406_7129_01_001453 for application appattempt_1495599791406_7129_000001 when currently reserved container container_e10_1495599791406_7123_01_001513 on node host: node0123:45454 #containers=40 available=... used=...
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode.reserveResource(FiCaSchedulerNode.java:81)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.reserve(FiCaSchedulerApp.java:1079)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.apply(FiCaSchedulerApp.java:795)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.tryCommit(CapacityScheduler.java:2770)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler$ResourceCommitterService.run(CapacityScheduler.java:546)
      

      Reproduce this problem:
      1. nm1 re-reserved app-1/container-X1 and generated reserve proposal-1
      2. nm2 had enough resource for app-1, un-reserved app-1/container-X1 and allocated app-1/container-X2
      3. nm1 reserved app-2/container-Y
      4. proposal-1 was accepted but throw IllegalStateException when applying

      Currently the check code for reserve proposal in FiCaSchedulerApp#accept as follows:

                // Container reserved first time will be NEW, after the container
                // accepted & confirmed, it will become RESERVED state
                if (schedulerContainer.getRmContainer().getState()
                    == RMContainerState.RESERVED) {
                  // Set reReservation == true
                  reReservation = true;
                } else {
                  // When reserve a resource (state == NEW is for new container,
                  // state == RUNNING is for increase container).
                  // Just check if the node is not already reserved by someone
                  if (schedulerContainer.getSchedulerNode().getReservedContainer()
                      != null) {
                    if (LOG.isDebugEnabled()) {
                      LOG.debug("Try to reserve a container, but the node is "
                          + "already reserved by another container="
                          + schedulerContainer.getSchedulerNode()
                          .getReservedContainer().getContainerId());
                    }
                    return false;
                  }
                }
      

      The reserved container on the node of reserve proposal will be checked only for first-reserve container.
      We should confirm that reserved container on this node is equal to re-reserve container.

      1. YARN-6678.001.patch
        14 kB
        Tao Yang
      2. YARN-6678.002.patch
        12 kB
        Tao Yang
      3. YARN-6678.003.patch
        12 kB
        Tao Yang
      4. YARN-6678.004.patch
        12 kB
        Tao Yang
      5. YARN-6678.005.patch
        11 kB
        Tao Yang
      6. YARN-6678.branch-2.005.patch
        11 kB
        Tao Yang

        Activity

        Hide
        Tao Yang Tao Yang added a comment -

        Attach a patch with UT for review.

        Show
        Tao Yang Tao Yang added a comment - Attach a patch with UT for review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 7s YARN-6678 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12870767/YARN-6678.001.patch
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16166/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s YARN-6678 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12870767/YARN-6678.001.patch Console output https://builds.apache.org/job/PreCommit-YARN-Build/16166/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 4s trunk passed
        +1 compile 0m 35s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 36s trunk passed
        +1 findbugs 1m 5s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 35s the patch passed
        +1 compile 0m 32s the patch passed
        +1 javac 0m 32s the patch passed
        -0 checkstyle 0m 28s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 294 unchanged - 0 fixed = 297 total (was 294)
        +1 mvnsite 0m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 4s the patch passed
        +1 javadoc 0m 19s the patch passed
        -1 unit 39m 1s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        61m 29s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873091/YARN-6678.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d9d76f41f383 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 315f077
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16185/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16185/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16185/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16185/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 4s trunk passed +1 compile 0m 35s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 36s trunk passed +1 findbugs 1m 5s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 35s the patch passed +1 compile 0m 32s the patch passed +1 javac 0m 32s the patch passed -0 checkstyle 0m 28s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 294 unchanged - 0 fixed = 297 total (was 294) +1 mvnsite 0m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 4s the patch passed +1 javadoc 0m 19s the patch passed -1 unit 39m 1s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 61m 29s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873091/YARN-6678.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d9d76f41f383 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 315f077 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16185/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16185/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16185/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16185/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 14s trunk passed
        +1 compile 0m 34s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 36s trunk passed
        +1 findbugs 0m 59s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 31s the patch passed
        +1 javac 0m 31s the patch passed
        -0 checkstyle 0m 27s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 294 unchanged - 0 fixed = 297 total (was 294)
        +1 mvnsite 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 3s the patch passed
        +1 javadoc 0m 19s the patch passed
        -1 unit 39m 1s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        60m 21s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873101/YARN-6678.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 084426ff1fb6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 315f077
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16186/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16186/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16186/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16186/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 14s trunk passed +1 compile 0m 34s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 36s trunk passed +1 findbugs 0m 59s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -0 checkstyle 0m 27s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 294 unchanged - 0 fixed = 297 total (was 294) +1 mvnsite 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 3s the patch passed +1 javadoc 0m 19s the patch passed -1 unit 39m 1s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 60m 21s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873101/YARN-6678.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 084426ff1fb6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 315f077 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16186/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16186/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16186/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16186/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Nice catching, thanks Tao Yang for patch and detailed analysis.

        Patch generally looks good, only one minor comment:

        Instead of using RmContainer().equals, it should be enough to use == to compare two instances, correct?

        Show
        leftnoteasy Wangda Tan added a comment - Nice catching, thanks Tao Yang for patch and detailed analysis. Patch generally looks good, only one minor comment: Instead of using RmContainer().equals , it should be enough to use == to compare two instances, correct?
        Hide
        leftnoteasy Wangda Tan added a comment -

        Sorry to add one more comment, Tao Yang, is there any other way to avoid adding the new method to CapacityScheduler?

        Show
        leftnoteasy Wangda Tan added a comment - Sorry to add one more comment, Tao Yang , is there any other way to avoid adding the new method to CapacityScheduler?
        Hide
        Tao Yang Tao Yang added a comment -

        Thanks Wangda Tan for your comments.

        Instead of using RmContainer().equals, it should be enough to use == to compare two instances, correct?

        Correct, just noticed that as you mentioned.

        is there any other way to avoid adding the new method to CapacityScheduler?

        It's necessary to add new method if spy on app attempt. I'll try to find another way to test this problem, for example, spy on CapacityScheduler instance

        Show
        Tao Yang Tao Yang added a comment - Thanks Wangda Tan for your comments. Instead of using RmContainer().equals, it should be enough to use == to compare two instances, correct? Correct, just noticed that as you mentioned. is there any other way to avoid adding the new method to CapacityScheduler? It's necessary to add new method if spy on app attempt. I'll try to find another way to test this problem, for example, spy on CapacityScheduler instance
        Hide
        leftnoteasy Wangda Tan added a comment -

        It's necessary to add new method if spy on app attempt. I'll try to find another way to test this problem, for example, spy on CapacityScheduler instance

        Or you can extend CapacityScheduler and add the method in the test case, it's better to avoid test-only method to production code.

        Show
        leftnoteasy Wangda Tan added a comment - It's necessary to add new method if spy on app attempt. I'll try to find another way to test this problem, for example, spy on CapacityScheduler instance Or you can extend CapacityScheduler and add the method in the test case, it's better to avoid test-only method to production code.
        Hide
        Tao Yang Tao Yang added a comment -

        Updated the patch without adding new method to CapacityScheduler. Thanks Wangda Tan for your suggestion, it's fine to only change the spy target for the test case.

        Show
        Tao Yang Tao Yang added a comment - Updated the patch without adding new method to CapacityScheduler. Thanks Wangda Tan for your suggestion, it's fine to only change the spy target for the test case.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 16m 4s trunk passed
        +1 compile 0m 35s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 38s trunk passed
        +1 findbugs 1m 6s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 33s the patch passed
        +1 compile 0m 32s the patch passed
        +1 javac 0m 32s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 34s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 8s the patch passed
        +1 javadoc 0m 18s the patch passed
        -1 unit 40m 30s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        64m 52s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
          hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874072/YARN-6678.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bbaa151c606c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b649519
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16222/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16222/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16222/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 16m 4s trunk passed +1 compile 0m 35s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 38s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 33s the patch passed +1 compile 0m 32s the patch passed +1 javac 0m 32s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 34s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 8s the patch passed +1 javadoc 0m 18s the patch passed -1 unit 40m 30s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 64m 52s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874072/YARN-6678.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bbaa151c606c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b649519 Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/16222/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16222/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16222/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        Thanks Tao Yang. Nice catch.

        Few comments:

        1. In FiCaSchedulerApp#accept, its better to use RMContainer#equals instead of using !=
        2. In same method's debug log, please add nodeId to log as well.
        3. In testcase named testCommitOutdatedReservedProposal, you can call rm.stop() at the end
        4. In same test case
          222	    while (true) {
          223	      if (sn1.getReservedContainer() != null) {
          224	        break;
          225	      }
          226	      Thread.sleep(100);
          227	    }
          

          I prefer to reduce timeout and do certain retries. Though test case has a timeout at top level, its better to make this 10ms and to 20 or 50 retries.

        Show
        sunilg Sunil G added a comment - Thanks Tao Yang . Nice catch. Few comments: In FiCaSchedulerApp#accept , its better to use RMContainer#equals instead of using != In same method's debug log, please add nodeId to log as well. In testcase named testCommitOutdatedReservedProposal , you can call rm.stop() at the end In same test case 222 while ( true ) { 223 if (sn1.getReservedContainer() != null ) { 224 break ; 225 } 226 Thread .sleep(100); 227 } I prefer to reduce timeout and do certain retries. Though test case has a timeout at top level, its better to make this 10ms and to 20 or 50 retries.
        Hide
        Tao Yang Tao Yang added a comment - - edited

        Thanks Sunil G for your comments.

        1. In FiCaSchedulerApp#accept, its better to use RMContainer#equals instead of using !=

        As Wangda Tan mentioned, it should be enough to use == to compare two instances. Are there some other concerns about this?
        I noticed that this patch caused several failed tests, but these are all passed when I run it locally. What might be the problem?

        Show
        Tao Yang Tao Yang added a comment - - edited Thanks Sunil G for your comments. 1. In FiCaSchedulerApp#accept, its better to use RMContainer#equals instead of using != As Wangda Tan mentioned, it should be enough to use == to compare two instances. Are there some other concerns about this? I noticed that this patch caused several failed tests, but these are all passed when I run it locally. What might be the problem?
        Hide
        sunilg Sunil G added a comment -

        We might need to check the error shown in TestCapacitySchedulerAsyncScheduling alone. Other test failures are not related to this patch. I ll also check this test case locally.

        Show
        sunilg Sunil G added a comment - We might need to check the error shown in TestCapacitySchedulerAsyncScheduling alone. Other test failures are not related to this patch. I ll also check this test case locally.
        Hide
        sunilg Sunil G added a comment -

        Kicked jenkins to see a clean a run again.

        Show
        sunilg Sunil G added a comment - Kicked jenkins to see a clean a run again.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 37s trunk passed
        +1 compile 0m 32s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 findbugs 0m 56s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 31s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 59s the patch passed
        +1 javadoc 0m 18s the patch passed
        -1 unit 43m 59s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        65m 36s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874072/YARN-6678.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 89756766d90e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e9d8bdf
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16267/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16267/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 37s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 35s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 59s the patch passed +1 javadoc 0m 18s the patch passed -1 unit 43m 59s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 65m 36s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874072/YARN-6678.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 89756766d90e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e9d8bdf Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/16267/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16267/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16267/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        I think we might need to debug more here as I am still getting test error.

        Show
        sunilg Sunil G added a comment - I think we might need to debug more here as I am still getting test error.
        Hide
        Tao Yang Tao Yang added a comment -

        Thanks Sunil G for your time.
        As your mentioned, This new patch adds timeout for every where clause, adds nodeId for debug info, and calls MockRM#stop at last of new test case. TestCapacitySchedulerAsyncScheduling can be passed now.
        Sorry to be late for updating this patch.

        Show
        Tao Yang Tao Yang added a comment - Thanks Sunil G for your time. As your mentioned, This new patch adds timeout for every where clause, adds nodeId for debug info, and calls MockRM#stop at last of new test case. TestCapacitySchedulerAsyncScheduling can be passed now. Sorry to be late for updating this patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 12s trunk passed
        +1 compile 0m 33s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 findbugs 0m 59s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 31s the patch passed
        +1 javac 0m 31s the patch passed
        -0 checkstyle 0m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 21 unchanged - 0 fixed = 23 total (was 21)
        +1 mvnsite 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 6s the patch passed
        +1 javadoc 0m 21s the patch passed
        -1 unit 44m 46s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        68m 3s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875237/YARN-6678.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b35be6cf30b0 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 3be2659
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16286/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16286/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16286/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16286/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 12s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 35s trunk passed +1 findbugs 0m 59s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -0 checkstyle 0m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 21 unchanged - 0 fixed = 23 total (was 21) +1 mvnsite 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 6s the patch passed +1 javadoc 0m 21s the patch passed -1 unit 44m 46s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 68m 3s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875237/YARN-6678.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b35be6cf30b0 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3be2659 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16286/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16286/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16286/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16286/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 53s trunk passed
        +1 compile 0m 34s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 39s trunk passed
        +1 findbugs 1m 8s trunk passed
        +1 javadoc 0m 24s trunk passed
        +1 mvninstall 0m 36s the patch passed
        +1 compile 0m 35s the patch passed
        +1 javac 0m 35s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 38s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 13s the patch passed
        +1 javadoc 0m 20s the patch passed
        -1 unit 48m 17s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 33s The patch does not generate ASF License warnings.
        71m 22s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
        Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands
          org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875318/YARN-6678.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 40b20a31d2ab 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / fa1aaee
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16290/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16290/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16290/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 53s trunk passed +1 compile 0m 34s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 39s trunk passed +1 findbugs 1m 8s trunk passed +1 javadoc 0m 24s trunk passed +1 mvninstall 0m 36s the patch passed +1 compile 0m 35s the patch passed +1 javac 0m 35s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 13s the patch passed +1 javadoc 0m 20s the patch passed -1 unit 48m 17s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 71m 22s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands   org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12875318/YARN-6678.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 40b20a31d2ab 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fa1aaee Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/16290/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16290/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16290/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        In last 2 runs and in local runs, I cannot see any more failures. Tao Yang, could you please confirm about the manual test result. If its fine, I can get this patch in.

        Show
        sunilg Sunil G added a comment - In last 2 runs and in local runs, I cannot see any more failures. Tao Yang , could you please confirm about the manual test result. If its fine, I can get this patch in.
        Hide
        Tao Yang Tao Yang added a comment -

        I confirmed that it's fine. Thanks Sunil G for your help.

        Show
        Tao Yang Tao Yang added a comment - I confirmed that it's fine. Thanks Sunil G for your help.
        Hide
        sunilg Sunil G added a comment -

        I think patch needs rebasing. Could you please help Tao Yang

        Show
        sunilg Sunil G added a comment - I think patch needs rebasing. Could you please help Tao Yang
        Hide
        Tao Yang Tao Yang added a comment -

        Sure, upload new patch to resolve the conflict with YARN-6714 in TestCapacitySchedulerAsyncScheduling.

        Show
        Tao Yang Tao Yang added a comment - Sure, upload new patch to resolve the conflict with YARN-6714 in TestCapacitySchedulerAsyncScheduling.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 13m 12s trunk passed
        +1 compile 0m 34s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 35s trunk passed
        +1 findbugs 1m 0s trunk passed
        +1 javadoc 0m 21s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 32s the patch passed
        +1 javac 0m 32s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 6s the patch passed
        +1 javadoc 0m 19s the patch passed
              Other Tests
        -1 unit 42m 27s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        63m 48s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue YARN-6678
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12877316/YARN-6678.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c03e686f4f3f 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 02b141a
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-YARN-Build/16458/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16458/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/16458/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 12s trunk passed +1 compile 0m 34s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 35s trunk passed +1 findbugs 1m 0s trunk passed +1 javadoc 0m 21s trunk passed       Patch Compile Tests +1 mvninstall 0m 32s the patch passed +1 compile 0m 32s the patch passed +1 javac 0m 32s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 6s the patch passed +1 javadoc 0m 19s the patch passed       Other Tests -1 unit 42m 27s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 63m 48s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6678 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12877316/YARN-6678.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c03e686f4f3f 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 02b141a Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-YARN-Build/16458/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16458/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/16458/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        sunilg Sunil G added a comment -

        Thanks Tao Yang,
        I will commit the patch in a day if there are no objections.

        Show
        sunilg Sunil G added a comment - Thanks Tao Yang , I will commit the patch in a day if there are no objections.
        Hide
        Tao Yang Tao Yang added a comment -

        Attached branch-2 patch for cleanly applying. Thanks Sunil G and Wangda Tan for commits and reviews.

        Show
        Tao Yang Tao Yang added a comment - Attached branch-2 patch for cleanly applying. Thanks Sunil G and Wangda Tan for commits and reviews.
        Hide
        sunilg Sunil G added a comment -

        Yes. Thanks Tao Yang
        Committing tomorrow if there are no objection.

        Show
        sunilg Sunil G added a comment - Yes. Thanks Tao Yang Committing tomorrow if there are no objection.
        Hide
        sunilg Sunil G added a comment -

        Committed to trunk/branch-2. Thank you Tao Yang for the contribution and thanks Wangda Tan for additional reviews.

        Show
        sunilg Sunil G added a comment - Committed to trunk/branch-2. Thank you Tao Yang for the contribution and thanks Wangda Tan for additional reviews.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12109 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12109/)
        YARN-6678. Handle IllegalStateException in Async Scheduling mode of (sunilg: rev f64cfeaf61ec65a465decdd8215f567d4e6677a9)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerAsyncScheduling.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12109 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12109/ ) YARN-6678 . Handle IllegalStateException in Async Scheduling mode of (sunilg: rev f64cfeaf61ec65a465decdd8215f567d4e6677a9) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerAsyncScheduling.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java

          People

          • Assignee:
            Tao Yang Tao Yang
            Reporter:
            Tao Yang Tao Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development