Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5537

Intermittent test failure of TestAMRMClient#testAMRMClientWithContainerResourceChange

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Refer to test report https://builds.apache.org/job/PreCommit-YARN-Build/12692/testReport/

      Running org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
      Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.018 sec <<< FAILURE! - in org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
      testAMRMClientWithContainerResourceChange(org.apache.hadoop.yarn.client.api.impl.TestAMRMClient)  Time elapsed: 1.183 sec  <<< FAILURE!
      java.lang.AssertionError: expected:<0> but was:<1>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.junit.Assert.assertEquals(Assert.java:542)
      	at org.apache.hadoop.yarn.client.api.impl.TestAMRMClient.doContainerResourceChange(TestAMRMClient.java:1019)
      	at org.apache.hadoop.yarn.client.api.impl.TestAMRMClient.testAMRMClientWithContainerResourceChange(TestAMRMClient.java:909)
      
      1. Failure_allocate.txt
        43 kB
        Bibin A Chundatt
      2. Failure.txt
        499 kB
        Bibin A Chundatt
      3. YARN-5537.0001.patch
        2 kB
        Bibin A Chundatt
      4. YARN-5537.0002.patch
        3 kB
        Bibin A Chundatt
      5. YARN-5537.0003.patch
        3 kB
        Bibin A Chundatt
      6. YARN-5537.0004.patch
        3 kB
        Bibin A Chundatt
      7. YARN-5537.0005.patch
        3 kB
        Bibin A Chundatt

        Activity

        Hide
        bibinchundatt Bibin A Chundatt added a comment - - edited

        Analysis
        AM container allocated to node localhost:43931 - container1
        container2,container3,container4 allocated to node localhost:36489

        Increase container request gets served once heartbeat is received from localhost:36489.
        During random failure node heartbeat request is received from node localhost:36489 during allocate request and increase request is served.

        Show
        bibinchundatt Bibin A Chundatt added a comment - - edited Analysis AM container allocated to node localhost:43931 - container1 container2,container3,container4 allocated to node localhost:36489 Increase container request gets served once heartbeat is received from localhost:36489 . During random failure node heartbeat request is received from node localhost:36489 during allocate request and increase request is served.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 22s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 12s the patch passed
        -1 unit 15m 57s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        28m 15s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824730/YARN-5537.0001.patch
        JIRA Issue YARN-5537
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux eaab4479470b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0faee62
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12841/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12841/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12841/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12841/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 0s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 12s the patch passed -1 unit 15m 57s hadoop-yarn-client in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 28m 15s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824730/YARN-5537.0001.patch JIRA Issue YARN-5537 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux eaab4479470b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0faee62 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/12841/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12841/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12841/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12841/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Failure not related to patch attached. Jira is already available for the same

        Show
        bibinchundatt Bibin A Chundatt added a comment - Failure not related to patch attached. Jira is already available for the same
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Could someone help me with review.

        Show
        bibinchundatt Bibin A Chundatt added a comment - Could someone help me with review.
        Hide
        varun_saxena Varun Saxena added a comment -

        Will have a look

        Show
        varun_saxena Varun Saxena added a comment - Will have a look
        Hide
        varun_saxena Varun Saxena added a comment -

        Thanks Bibin A Chundatt for the patch. Analysis is correct.
        The fix will work but I do see a lot of deterministic sleeps in the test. This however becomes necessary because there is no way to drain multiple dispatchers across multiple NMs' and RM in MiniYarnCluster.

        For the extra sleeps we have added to fix the test, can we have them in a loop and exit whenever the values for increased and decreased containers have reached the required number or not OR after a fixed number of times, whichever is earlier.
        This would mean that test would not have sleep extra time if not required. Pls note this test passes more than it fails.

        Show
        varun_saxena Varun Saxena added a comment - Thanks Bibin A Chundatt for the patch. Analysis is correct. The fix will work but I do see a lot of deterministic sleeps in the test. This however becomes necessary because there is no way to drain multiple dispatchers across multiple NMs' and RM in MiniYarnCluster. For the extra sleeps we have added to fix the test, can we have them in a loop and exit whenever the values for increased and decreased containers have reached the required number or not OR after a fixed number of times, whichever is earlier. This would mean that test would not have sleep extra time if not required. Pls note this test passes more than it fails.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 9s trunk passed
        +1 compile 0m 26s trunk passed
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 32s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 0m 36s trunk passed
        +1 javadoc 0m 18s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 15s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 40s the patch passed
        +1 javadoc 0m 14s the patch passed
        -1 unit 16m 23s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        31m 59s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825172/YARN-5537.0002.patch
        JIRA Issue YARN-5537
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 22e07e41344b 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c37346d
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12870/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12870/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12870/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12870/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 9s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 36s trunk passed +1 javadoc 0m 18s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 40s the patch passed +1 javadoc 0m 14s the patch passed -1 unit 16m 23s hadoop-yarn-client in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 31m 59s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825172/YARN-5537.0002.patch JIRA Issue YARN-5537 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 22e07e41344b 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c37346d Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/12870/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12870/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12870/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12870/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Attaching patch after minor update

        Show
        bibinchundatt Bibin A Chundatt added a comment - Attaching patch after minor update
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 48s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 25s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 17s the patch passed
        +1 javac 0m 17s the patch passed
        -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 56 unchanged - 0 fixed = 57 total (was 56)
        +1 mvnsite 0m 21s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 12s the patch passed
        -1 unit 15m 56s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        27m 53s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825278/YARN-5537.0003.patch
        JIRA Issue YARN-5537
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0eb7354d3cad 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6fa9bf4
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12876/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12876/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 48s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed -1 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 1 new + 56 unchanged - 0 fixed = 57 total (was 56) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 12s the patch passed -1 unit 15m 56s hadoop-yarn-client in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 27m 53s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825278/YARN-5537.0003.patch JIRA Issue YARN-5537 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0eb7354d3cad 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6fa9bf4 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12876/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12876/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12876/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Uploading patch after checkstyle update

        Show
        bibinchundatt Bibin A Chundatt added a comment - Uploading patch after checkstyle update
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 54s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 25s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 19s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 21s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 33s the patch passed
        +1 javadoc 0m 11s the patch passed
        -1 unit 15m 58s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        28m 5s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825295/YARN-5537.0004.patch
        JIRA Issue YARN-5537
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5769a7e15e0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6fa9bf4
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12877/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12877/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12877/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12877/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 54s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 25s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 33s the patch passed +1 javadoc 0m 11s the patch passed -1 unit 15m 58s hadoop-yarn-client in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 28m 5s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825295/YARN-5537.0004.patch JIRA Issue YARN-5537 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5769a7e15e0e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6fa9bf4 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/12877/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12877/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12877/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12877/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        varun_saxena Varun Saxena added a comment -

        Unit test failure is unrelated.
        Will commit this later today.

        Show
        varun_saxena Varun Saxena added a comment - Unit test failure is unrelated. Will commit this later today.
        Hide
        sunilg Sunil G added a comment -

        Hi Bibin A Chundatt
        Thanks for the patch and approach looks fine and patch also overall looks fine. A minor nit, could we make the sleep deterministic. Like in MockRM, we can add a retry limit for sleep. Refer MockRM#waitForState. This may look more better. and each round we can sleep for few milli seconds till we reach a max limit. Thoughts?

        Show
        sunilg Sunil G added a comment - Hi Bibin A Chundatt Thanks for the patch and approach looks fine and patch also overall looks fine. A minor nit, could we make the sleep deterministic. Like in MockRM, we can add a retry limit for sleep. Refer MockRM#waitForState . This may look more better. and each round we can sleep for few milli seconds till we reach a max limit. Thoughts?
        Hide
        varun_saxena Varun Saxena added a comment -

        and each round we can sleep for few milli seconds till we reach a max limit.

        Sunil G, that is what is done in the patch. We sleep within while loop and break when the required condition is met.
        Or do you mean something else ?

        MiniYarnCluster tests typically do not check for internal states for RM and NM so I am not sure if we can have any generic methods in MiniYarnCluster for specific conditions.
        They are more for E2E flow testing.

        Show
        varun_saxena Varun Saxena added a comment - and each round we can sleep for few milli seconds till we reach a max limit. Sunil G , that is what is done in the patch. We sleep within while loop and break when the required condition is met. Or do you mean something else ? MiniYarnCluster tests typically do not check for internal states for RM and NM so I am not sure if we can have any generic methods in MiniYarnCluster for specific conditions. They are more for E2E flow testing.
        Hide
        sunilg Sunil G added a comment -

        I guess I have phrased my comment incorrectly by seeing a part of code. Could we have a small sleep inside than 100ms? Do we really need that much? May be we can increase counts with lesser sleep duration.

        Show
        sunilg Sunil G added a comment - I guess I have phrased my comment incorrectly by seeing a part of code. Could we have a small sleep inside than 100ms? Do we really need that much? May be we can increase counts with lesser sleep duration.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Could we have a small sleep inside than 100ms.

        Node manager heartbeat interval is ~100ms so really.Only in few cases its can go about 3 attempts. Should be fine i guess

        Show
        bibinchundatt Bibin A Chundatt added a comment - Could we have a small sleep inside than 100ms. Node manager heartbeat interval is ~100ms so really.Only in few cases its can go about 3 attempts. Should be fine i guess
        Hide
        varun_saxena Varun Saxena added a comment -

        Well, a smaller time slice can reduce the time by a bit (albeit in this case only some tens of milliseconds) as Node HB comes every 100 ms. Maybe it can be changed to a lower value.

        Show
        varun_saxena Varun Saxena added a comment - Well, a smaller time slice can reduce the time by a bit (albeit in this case only some tens of milliseconds) as Node HB comes every 100 ms. Maybe it can be changed to a lower value.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Attaching patch after changing sleep to 20 and and iterations to 30

        Show
        bibinchundatt Bibin A Chundatt added a comment - Attaching patch after changing sleep to 20 and and iterations to 30
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 44s trunk passed
        +1 compile 0m 20s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 24s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 14s trunk passed
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 16s the patch passed
        +1 javac 0m 16s the patch passed
        +1 checkstyle 0m 12s the patch passed
        +1 mvnsite 0m 22s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 43s the patch passed
        +1 javadoc 0m 14s the patch passed
        -1 unit 16m 19s hadoop-yarn-client in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        28m 33s



        Reason Tests
        Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825419/YARN-5537.0005.patch
        JIRA Issue YARN-5537
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9180c829696c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ab3b727
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/12889/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12889/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12889/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12889/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 44s trunk passed +1 compile 0m 20s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 18s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 43s the patch passed +1 javadoc 0m 14s the patch passed -1 unit 16m 19s hadoop-yarn-client in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 28m 33s Reason Tests Failed junit tests hadoop.yarn.client.api.impl.TestYarnClient Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825419/YARN-5537.0005.patch JIRA Issue YARN-5537 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9180c829696c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ab3b727 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/12889/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12889/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12889/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Console output https://builds.apache.org/job/PreCommit-YARN-Build/12889/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        varun_saxena Varun Saxena added a comment -

        Committed to trunk, branch-2.
        Thanks Bibin A Chundatt for your contribution and Sunil G for reviews.

        Show
        varun_saxena Varun Saxena added a comment - Committed to trunk, branch-2. Thanks Bibin A Chundatt for your contribution and Sunil G for reviews.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10342 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10342/)
        YARN-5537. Fix intermittent failure of (varunsaxena: rev 79603f588238d47c77748682add0502fb89bbf48)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClient.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10342 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10342/ ) YARN-5537 . Fix intermittent failure of (varunsaxena: rev 79603f588238d47c77748682add0502fb89bbf48) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClient.java

          People

          • Assignee:
            bibinchundatt Bibin A Chundatt
            Reporter:
            varun_saxena Varun Saxena
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development