Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5377

Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      Test case fails jenkin build link

      Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 134.586 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
      testKillMultipleOpportunisticContainers(org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager)  Time elapsed: 32.134 sec  <<< FAILURE!
      java.lang.AssertionError: ContainerState is not correct (timedout) expected:<DONE> but was:<CONTAINER_CLEANEDUP_AFTER_KILL>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.BaseContainerManagerTest.waitForNMContainerState(BaseContainerManagerTest.java:363)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager.testKillMultipleOpportunisticContainers(TestQueuingContainerManager.java:470)
      
      1. YARN-5377.001.patch
        6 kB
        Konstantinos Karanasos

        Issue Links

          Activity

          Hide
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

          This happened again in YARN-5725 at one of the patches. I was not able to reproduce it locally.

          Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 134.743 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          testKillMultipleOpportunisticContainers(org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager)  Time elapsed: 32.169 sec  <<< FAILURE!
          java.lang.AssertionError: ContainerState is not correct (timedout) expected:<DONE> but was:<CONTAINER_CLEANEDUP_AFTER_KILL>
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.failNotEquals(Assert.java:743)
          	at org.junit.Assert.assertEquals(Assert.java:118)
          	at org.apache.hadoop.yarn.server.nodemanager.containermanager.BaseContainerManagerTest.waitForNMContainerState(BaseContainerManagerTest.java:368)
          	at org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager.testKillMultipleOpportunisticContainers(TestQueuingContainerManager.java:470)
          

          I am wondering about the root cause since the timeout is already 40 seconds.

              BaseContainerManagerTest.waitForNMContainerState(containerManager,
                  createContainerId(0), ContainerState.DONE, 40);
          
          Show
          miklos.szegedi@cloudera.com Miklos Szegedi added a comment - This happened again in YARN-5725 at one of the patches. I was not able to reproduce it locally. Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 134.743 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager testKillMultipleOpportunisticContainers(org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager) Time elapsed: 32.169 sec <<< FAILURE! java.lang.AssertionError: ContainerState is not correct (timedout) expected:<DONE> but was:<CONTAINER_CLEANEDUP_AFTER_KILL> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.apache.hadoop.yarn.server.nodemanager.containermanager.BaseContainerManagerTest.waitForNMContainerState(BaseContainerManagerTest.java:368) at org.apache.hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager.testKillMultipleOpportunisticContainers(TestQueuingContainerManager.java:470) I am wondering about the root cause since the timeout is already 40 seconds. BaseContainerManagerTest.waitForNMContainerState(containerManager, createContainerId(0), ContainerState.DONE, 40);
          Hide
          asuresh Arun Suresh added a comment - - edited

          Assigning this to Konstantinos Karanasos, since he's the original author of the testcase and he has a patch available for this.

          Show
          asuresh Arun Suresh added a comment - - edited Assigning this to Konstantinos Karanasos , since he's the original author of the testcase and he has a patch available for this.
          Hide
          kkaranasos Konstantinos Karanasos added a comment - - edited

          The problem with the test is that the container was moving fast from the DONE to the CONTAINER_CLEANUP_AFTER_KILL state, and the DONE state was not observed by the waitForNMContainerState method of the BaseContainerManagerTest.

          I added a new waitForNMContainerState method that takes as input a list of final container states, instead of a single one like before. When any of the states of this list is reached, the waitForNMContainerState exits successfully.

          Show
          kkaranasos Konstantinos Karanasos added a comment - - edited The problem with the test is that the container was moving fast from the DONE to the CONTAINER_CLEANUP_AFTER_KILL state, and the DONE state was not observed by the waitForNMContainerState method of the BaseContainerManagerTest . I added a new waitForNMContainerState method that takes as input a list of final container states, instead of a single one like before. When any of the states of this list is reached, the waitForNMContainerState exits successfully.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 7m 25s trunk passed
          +1 compile 0m 32s trunk passed
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 29s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 43s trunk passed
          +1 javadoc 0m 17s trunk passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 25s the patch passed
          +1 javac 0m 25s the patch passed
          +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 0 new + 32 unchanged - 2 fixed = 32 total (was 34)
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 46s the patch passed
          +1 javadoc 0m 14s the patch passed
          +1 unit 14m 29s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          28m 48s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-5377
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836999/YARN-5377.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 23cf1854165d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5cad93d
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13775/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13775/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 25s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 43s trunk passed +1 javadoc 0m 17s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 checkstyle 0m 13s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 0 new + 32 unchanged - 2 fixed = 32 total (was 34) +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 46s the patch passed +1 javadoc 0m 14s the patch passed +1 unit 14m 29s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 28m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-5377 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836999/YARN-5377.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 23cf1854165d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5cad93d Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13775/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13775/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          asuresh Arun Suresh added a comment -

          Thanks for reporting this Rohith Sharma K S, Miklos Szegedi and for the patch Konstantinos Karanasos
          +1, Committing this shortly

          Show
          asuresh Arun Suresh added a comment - Thanks for reporting this Rohith Sharma K S , Miklos Szegedi and for the patch Konstantinos Karanasos +1, Committing this shortly
          Hide
          asuresh Arun Suresh added a comment -

          Committed this to trunk

          Show
          asuresh Arun Suresh added a comment - Committed this to trunk
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10787 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10787/)
          YARN-5377. Fix (arun suresh: rev f38a6d03a11ca6de93a225563ddf55ec99d5063c)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/queuing/TestQueuingContainerManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10787 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10787/ ) YARN-5377 . Fix (arun suresh: rev f38a6d03a11ca6de93a225563ddf55ec99d5063c) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/queuing/TestQueuingContainerManager.java
          Hide
          asuresh Arun Suresh added a comment -

          Committing this to branch-2

          Show
          asuresh Arun Suresh added a comment - Committing this to branch-2

            People

            • Assignee:
              kkaranasos Konstantinos Karanasos
              Reporter:
              rohithsharma Rohith Sharma K S
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development