Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3992

TestApplicationPriority.testApplicationPriorityAllocation fails intermittently

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      java.lang.AssertionError: expected:<7> but was:<5>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.junit.Assert.assertEquals(Assert.java:542)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority.testApplicationPriorityAllocation(TestApplicationPriority.java:182)
      
      1. 0003-YARN-3992.patch
        9 kB
        Sunil G
      2. 0002-YARN-3992.patch
        12 kB
        Sunil G
      3. 0001-YARN-3992.patch
        7 kB
        Sunil G

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2225 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2225/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2225 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2225/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/276/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #276 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/276/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #268 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/268/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #268 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/268/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2206 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2206/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2206 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2206/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #279 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/279/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #279 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/279/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1009 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1009/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1009 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1009/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Hide
          sunilg Sunil G added a comment -

          Thank you Rohith Sharma K S for the review and commit.!

          Show
          sunilg Sunil G added a comment - Thank you Rohith Sharma K S for the review and commit.!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8269 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8269/)
          YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8269 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8269/ ) YARN-3992 . TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) (rohithsharmaks: rev df9e7280db58baddd02d6e23d3685efb8d5f1b97) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationPriority.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Committed to branch-2/trunk. Thanks Sunil G !!

          Show
          rohithsharma Rohith Sharma K S added a comment - Committed to branch-2/trunk. Thanks Sunil G !!
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          +1 lgtm

          Show
          rohithsharma Rohith Sharma K S added a comment - +1 lgtm
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 7m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 8m 11s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 50s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 24s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 54m 20s Tests passed in hadoop-yarn-server-resourcemanager.
              74m 29s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748870/0003-YARN-3992.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 52f3525
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8772/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8772/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8772/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 7m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 11s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 50s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 24s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 54m 20s Tests passed in hadoop-yarn-server-resourcemanager.     74m 29s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748870/0003-YARN-3992.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 52f3525 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8772/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8772/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8772/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Thanks Rohith Sharma K S. Change method signature so that existing test cases can use old method itself.

          Show
          sunilg Sunil G added a comment - Thanks Rohith Sharma K S . Change method signature so that existing test cases can use old method itself.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          The patch looks overall good,
          nit: Can you add new API with additional parameter for host instead of changing existing API allocateAndWaitForContainers arguments?

          Show
          rohithsharma Rohith Sharma K S added a comment - The patch looks overall good, nit: Can you add new API with additional parameter for host instead of changing existing API allocateAndWaitForContainers arguments?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 5m 55s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 7m 55s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 26s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 yarn tests 52m 26s Tests failed in hadoop-yarn-server-resourcemanager.
              70m 24s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748287/0002-YARN-3992.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / d311a38
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8749/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8749/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8749/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 5m 55s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 55s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 26s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 findbugs 1m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 yarn tests 52m 26s Tests failed in hadoop-yarn-server-resourcemanager.     70m 24s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748287/0002-YARN-3992.patch Optional Tests javac unit findbugs checkstyle git revision trunk / d311a38 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8749/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8749/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8749/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Thank you Rohith Sharma K S for pointing out this.
          A slight modification is done in the allocateAndWaitForContainers api to pass host also. Please help to review the same.

          Show
          sunilg Sunil G added a comment - Thank you Rohith Sharma K S for pointing out this. A slight modification is done in the allocateAndWaitForContainers api to pass host also. Please help to review the same.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Sunil G for providing the patch!!
          One comment

          1. Instead of rewritting below code twice, can you use method MockAM#allocateAndWaitForContainers so many lines of code can be avoided.
            +    int NUM_CONTAINERS = 7;
            +    // allocate NUM_CONTAINERS containers
            +    am1.allocate("127.0.0.1", 2 * GB, NUM_CONTAINERS,
            +        new ArrayList<ContainerId>());
                 nm1.nodeHeartbeat(true);
            -    while (alloc1Response.getAllocatedContainers().size() < 1) {
            -      LOG.info("Waiting for containers to be created for app 1...");
            -      Thread.sleep(100);
            -      alloc1Response = am1.schedule();
            +
            +    // wait for containers to be allocated.
            +    List<Container> allocated1 = am1.allocate(new ArrayList<ResourceRequest>(),
            +        new ArrayList<ContainerId>()).getAllocatedContainers();
            +    while (allocated1.size() != NUM_CONTAINERS) {
            +      nm1.nodeHeartbeat(true);
            +      allocated1.addAll(am1.allocate(new ArrayList<ResourceRequest>(),
            +          new ArrayList<ContainerId>()).getAllocatedContainers());
            +      Thread.sleep(200);
                 }
            
          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Sunil G for providing the patch!! One comment Instead of rewritting below code twice, can you use method MockAM#allocateAndWaitForContainers so many lines of code can be avoided. + int NUM_CONTAINERS = 7; + // allocate NUM_CONTAINERS containers + am1.allocate( "127.0.0.1" , 2 * GB, NUM_CONTAINERS, + new ArrayList<ContainerId>()); nm1.nodeHeartbeat( true ); - while (alloc1Response.getAllocatedContainers().size() < 1) { - LOG.info( "Waiting for containers to be created for app 1..." ); - Thread .sleep(100); - alloc1Response = am1.schedule(); + + // wait for containers to be allocated. + List<Container> allocated1 = am1.allocate( new ArrayList<ResourceRequest>(), + new ArrayList<ContainerId>()).getAllocatedContainers(); + while (allocated1.size() != NUM_CONTAINERS) { + nm1.nodeHeartbeat( true ); + allocated1.addAll(am1.allocate( new ArrayList<ResourceRequest>(), + new ArrayList<ContainerId>()).getAllocatedContainers()); + Thread .sleep(200); }
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 6m 19s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 11s There were no new javac warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 29s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 53m 7s Tests passed in hadoop-yarn-server-resourcemanager.
              71m 52s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747833/0001-YARN-3992.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / c020b62
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8708/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8708/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8708/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 6m 19s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 11s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 29s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 27s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 53m 7s Tests passed in hadoop-yarn-server-resourcemanager.     71m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747833/0001-YARN-3992.patch Optional Tests javac unit findbugs checkstyle git revision trunk / c020b62 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8708/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8708/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8708/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Thank you Zhijie Shen. It seems we were not waiting for full containers to get allocated. I have now updated code so that we wait for all containers to get allocated.

          Show
          sunilg Sunil G added a comment - Thank you Zhijie Shen . It seems we were not waiting for full containers to get allocated. I have now updated code so that we wait for all containers to get allocated.
          Hide
          sunilg Sunil G added a comment -

          Thank you Zhijie Shen for reporting the same. I will take a look into this.

          Show
          sunilg Sunil G added a comment - Thank you Zhijie Shen for reporting the same. I will take a look into this.
          Hide
          sunilg Sunil G added a comment -

          Thank you [~zhi

          Show
          sunilg Sunil G added a comment - Thank you [~zhi
          Hide
          zjshen Zhijie Shen added a comment -
          Show
          zjshen Zhijie Shen added a comment - The problem was found with jenkins build on YARN-3049 : https://builds.apache.org/job/PreCommit-YARN-Build/8701/testReport/

            People

            • Assignee:
              sunilg Sunil G
              Reporter:
              zjshen Zhijie Shen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development