Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-3445

Cache runningApps in RMNode for getting running apps on given NodeId

    Details

    • Target Version/s:

      Description

      Per discussion in YARN-3334, we need filter out unnecessary collectors info from RM in heartbeat response. Our propose is to add cache for runningApps in RMNode, so RM only send collectors for local running apps back. This is also needed in YARN-914 (graceful decommission) that if no running apps in NM which is in decommissioning stage, it will get decommissioned immediately.

      1. YARN-3445.patch
        11 kB
        Junping Du
      2. YARN-3445-v2.patch
        10 kB
        Junping Du
      3. YARN-3445-v3.1.patch
        12 kB
        Junping Du
      4. YARN-3445-v3.patch
        12 kB
        Junping Du
      5. YARN-3445-v4.1.patch
        12 kB
        Junping Du
      6. YARN-3445-v4.patch
        12 kB
        Junping Du
      7. YARN-3445-v5.1.patch
        11 kB
        Junping Du
      8. YARN-3445-v5.patch
        12 kB
        Junping Du

        Issue Links

          Activity

          Hide
          djp Junping Du added a comment -

          Upload a patch.

          Show
          djp Junping Du added a comment - Upload a patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726589/YARN-3445.patch
          against trunk revision c17cd4f.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7398//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7398//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726589/YARN-3445.patch against trunk revision c17cd4f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7398//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7398//console This message is automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          This doesn't sound right. RM already knows what is running and what isn't. Why is NM informing the RM?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - This doesn't sound right. RM already knows what is running and what isn't. Why is NM informing the RM?
          Hide
          djp Junping Du added a comment -

          This doesn't sound right. RM already knows what is running and what isn't. Why is NM informing the RM?

          I could be mis-leaded by RegisterNodeManagerRequest when NM register to RM (for RM restart I think). However, it seems like there is no quick or handy way to do NodeId -> runningApps. The existing info there should be RMAppImpl -> ranNodes only. May be we need to setup reverse mapping somewhere instead - e.g. track runningApplication in RMNodeImpl?

          Show
          djp Junping Du added a comment - This doesn't sound right. RM already knows what is running and what isn't. Why is NM informing the RM? I could be mis-leaded by RegisterNodeManagerRequest when NM register to RM (for RM restart I think). However, it seems like there is no quick or handy way to do NodeId -> runningApps. The existing info there should be RMAppImpl -> ranNodes only. May be we need to setup reverse mapping somewhere instead - e.g. track runningApplication in RMNodeImpl?
          Hide
          djp Junping Du added a comment -

          I have updated the title and description. Hi Vinod Kumar Vavilapalli, does this sounds right to you now?

          Show
          djp Junping Du added a comment - I have updated the title and description. Hi Vinod Kumar Vavilapalli , does this sounds right to you now?
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Better than before, will comment once see an updated patch.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Better than before, will comment once see an updated patch.
          Hide
          djp Junping Du added a comment -

          Update patch in v2 with new way proposed above.

          Show
          djp Junping Du added a comment - Update patch in v2 with new way proposed above.
          Hide
          hadoopqa Hadoop QA added a comment -

          The patch artifact directory on has been removed!
          This is a fatal error for test-patch.sh. Aborting.
          Jenkins (node H3) information at https://builds.apache.org/job/PreCommit-YARN-Build/7525/ may provide some hints.

          Show
          hadoopqa Hadoop QA added a comment - The patch artifact directory on has been removed! This is a fatal error for test-patch.sh. Aborting. Jenkins (node H3) information at https://builds.apache.org/job/PreCommit-YARN-Build/7525/ may provide some hints.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 34s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 whitespace 0m 0s The patch has 15 line(s) that end in whitespace.
          +1 javac 7m 31s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 5m 20s The applied patch generated 4 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          -1 findbugs 1m 59s The patch appears to introduce 14 new Findbugs (version 2.0.3) warnings.
          +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls.
          +1 yarn tests 52m 26s Tests passed in hadoop-yarn-server-resourcemanager.
              95m 0s  



          Reason Tests
          FindBugs module:hadoop-sls
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String): new java.io.FileReader(String) At RumenToSLSConverter.java:[line 122]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String): new java.io.FileWriter(String) At RumenToSLSConverter.java:[line 124]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSNodeFile(String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSNodeFile(String): new java.io.FileWriter(String) At RumenToSLSConverter.java:[line 145]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.SLSRunner.startAMFromSLSTraces(Resource, int):in org.apache.hadoop.yarn.sls.SLSRunner.startAMFromSLSTraces(Resource, int): new java.io.FileReader(String) At SLSRunner.java:[line 280]
            Unwritten field:NodeInfo.java:[line 140]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics():in org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(): new java.io.FileWriter(String) At ResourceSchedulerWrapper.java:[line 490]
            Found reliance on default encoding in new org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper$MetricsLogRunnable(ResourceSchedulerWrapper):in new org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper$MetricsLogRunnable(ResourceSchedulerWrapper): new java.io.FileWriter(String) At ResourceSchedulerWrapper.java:[line 695]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.initMetrics():in org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.initMetrics(): new java.io.FileWriter(String) At SLSCapacityScheduler.java:[line 493]
            Found reliance on default encoding in new org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler$MetricsLogRunnable(SLSCapacityScheduler):in new org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler$MetricsLogRunnable(SLSCapacityScheduler): new java.io.FileWriter(String) At SLSCapacityScheduler.java:[line 698]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromNodeFile(String):in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromNodeFile(String): new java.io.FileReader(String) At SLSUtils.java:[line 119]
            Found reliance on default encoding in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(String):in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(String): new java.io.FileReader(String) At SLSUtils.java:[line 92]
            Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field handleOperTimecostHistogramMap In SLSWebApp.java:instance field handleOperTimecostHistogramMap In SLSWebApp.java
            Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field queueAllocatedMemoryCounterMap In SLSWebApp.java:instance field queueAllocatedMemoryCounterMap In SLSWebApp.java
            Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field queueAllocatedVCoresCounterMap In SLSWebApp.java:instance field queueAllocatedVCoresCounterMap In SLSWebApp.java



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728844/YARN-3445-v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 8f82970
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/whitespace.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/checkstyle-result-diff.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7539/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7539/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 34s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 whitespace 0m 0s The patch has 15 line(s) that end in whitespace. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 20s The applied patch generated 4 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. -1 findbugs 1m 59s The patch appears to introduce 14 new Findbugs (version 2.0.3) warnings. +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls. +1 yarn tests 52m 26s Tests passed in hadoop-yarn-server-resourcemanager.     95m 0s   Reason Tests FindBugs module:hadoop-sls   Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String): new java.io.FileReader(String) At RumenToSLSConverter.java: [line 122]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSLoadFile(String, String): new java.io.FileWriter(String) At RumenToSLSConverter.java: [line 124]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSNodeFile(String):in org.apache.hadoop.yarn.sls.RumenToSLSConverter.generateSLSNodeFile(String): new java.io.FileWriter(String) At RumenToSLSConverter.java: [line 145]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.SLSRunner.startAMFromSLSTraces(Resource, int):in org.apache.hadoop.yarn.sls.SLSRunner.startAMFromSLSTraces(Resource, int): new java.io.FileReader(String) At SLSRunner.java: [line 280]   Unwritten field:NodeInfo.java: [line 140]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics():in org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(): new java.io.FileWriter(String) At ResourceSchedulerWrapper.java: [line 490]   Found reliance on default encoding in new org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper$MetricsLogRunnable(ResourceSchedulerWrapper):in new org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper$MetricsLogRunnable(ResourceSchedulerWrapper): new java.io.FileWriter(String) At ResourceSchedulerWrapper.java: [line 695]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.initMetrics():in org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler.initMetrics(): new java.io.FileWriter(String) At SLSCapacityScheduler.java: [line 493]   Found reliance on default encoding in new org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler$MetricsLogRunnable(SLSCapacityScheduler):in new org.apache.hadoop.yarn.sls.scheduler.SLSCapacityScheduler$MetricsLogRunnable(SLSCapacityScheduler): new java.io.FileWriter(String) At SLSCapacityScheduler.java: [line 698]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromNodeFile(String):in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromNodeFile(String): new java.io.FileReader(String) At SLSUtils.java: [line 119]   Found reliance on default encoding in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(String):in org.apache.hadoop.yarn.sls.utils.SLSUtils.parseNodesFromSLSTrace(String): new java.io.FileReader(String) At SLSUtils.java: [line 92]   Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field handleOperTimecostHistogramMap In SLSWebApp.java:instance field handleOperTimecostHistogramMap In SLSWebApp.java   Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field queueAllocatedMemoryCounterMap In SLSWebApp.java:instance field queueAllocatedMemoryCounterMap In SLSWebApp.java   Class org.apache.hadoop.yarn.sls.web.SLSWebApp defines non-transient non-serializable instance field queueAllocatedVCoresCounterMap In SLSWebApp.java:instance field queueAllocatedVCoresCounterMap In SLSWebApp.java Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728844/YARN-3445-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f82970 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/whitespace.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/checkstyle-result-diff.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7539/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7539/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7539/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          There is a too much of duplicate information already in NodeHeartbeatRequest, albeit for slightly different purposes. We need to consolidate the following (without breaking compatibility of previous releases), lest the heartbeat will become heavier and heavier.

          • logAggregationReportsForApps added, but not released yet
            • logAggregationReportsForApps itself is a map of ApplicationID with a nested LogAggregationReport.ApplicationID - duplicate AppID information
          • runningApplications in this patch
          • NodeStatus.keepAliveApplications

          /cc Jian He Wangda Tan

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - There is a too much of duplicate information already in NodeHeartbeatRequest, albeit for slightly different purposes. We need to consolidate the following (without breaking compatibility of previous releases), lest the heartbeat will become heavier and heavier. logAggregationReportsForApps added, but not released yet logAggregationReportsForApps itself is a map of ApplicationID with a nested LogAggregationReport.ApplicationID - duplicate AppID information runningApplications in this patch NodeStatus.keepAliveApplications /cc Jian He Wangda Tan
          Hide
          djp Junping Du added a comment -

          Thanks for comments, Vinod Kumar Vavilapalli!

          logAggregationReportsForApps itself is a map of ApplicationID with a nested LogAggregationReport.ApplicationID - duplicate AppID information

          Are u suggest we should replace map with list in NodeHeartbeatRequest? I fully agree and I will suggest to do so in YARN-3505.

          runningApplications in this patch

          In v2 patch, runningApplications is already removed. Kindly check the v2 patch again?

          NodeStatus.keepAliveApplications

          I agree. This shouldn't be needed anymore after YARN-1402. I had the similar idea before in synced with Xuan but forget to put it on JIRA. May be we should file a separated JIRA to fix it?

          CC Xuan Gong.

          Show
          djp Junping Du added a comment - Thanks for comments, Vinod Kumar Vavilapalli ! logAggregationReportsForApps itself is a map of ApplicationID with a nested LogAggregationReport.ApplicationID - duplicate AppID information Are u suggest we should replace map with list in NodeHeartbeatRequest? I fully agree and I will suggest to do so in YARN-3505 . runningApplications in this patch In v2 patch, runningApplications is already removed. Kindly check the v2 patch again? NodeStatus.keepAliveApplications I agree. This shouldn't be needed anymore after YARN-1402 . I had the similar idea before in synced with Xuan but forget to put it on JIRA. May be we should file a separated JIRA to fix it? CC Xuan Gong .
          Hide
          jianhe Jian He added a comment -

          One other thing, the LogAggregationReport#(get/set)getNodeId can also be removed as it's not used anywhere.
          I'm also unsure about the usage of LogAggregationReport#(get/set)DiagnosticMessage as it's only set with an empty string.

          agree we can have a separate jira to fix this, preferably in the same 2.8 release.

          Show
          jianhe Jian He added a comment - One other thing, the LogAggregationReport#(get/set)getNodeId can also be removed as it's not used anywhere. I'm also unsure about the usage of LogAggregationReport#(get/set)DiagnosticMessage as it's only set with an empty string. agree we can have a separate jira to fix this, preferably in the same 2.8 release.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          I agree. This shouldn't be needed anymore after YARN-1402. I had the similar idea before in synced with Xuan but forget to put it on JIRA. May be we should file a separated JIRA to fix it?

          keepAliveApplications cannot be removed as we need to support protocol compatibility. But the new ones you added for logs can be removed as they are new. Can you take this forward also on YARN-3505?

          .. LogAggregationReport#(get/set)getNodeId .. LogAggregationReport#(get/set)DiagnosticMessage ..

          Jian He, can you file a ticket please?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - I agree. This shouldn't be needed anymore after YARN-1402 . I had the similar idea before in synced with Xuan but forget to put it on JIRA. May be we should file a separated JIRA to fix it? keepAliveApplications cannot be removed as we need to support protocol compatibility. But the new ones you added for logs can be removed as they are new. Can you take this forward also on YARN-3505 ? .. LogAggregationReport#(get/set)getNodeId .. LogAggregationReport#(get/set)DiagnosticMessage .. Jian He , can you file a ticket please?
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Jian He, can you file a ticket please?

          Actually, we can do this too on YARN-3505 as that is related to LogAggregationReport. Please leave a comment there..

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Jian He, can you file a ticket please? Actually, we can do this too on YARN-3505 as that is related to LogAggregationReport. Please leave a comment there..
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          With those comments out of the way, the patch looks good to me. But I'd still like to see a follow up JIRA and a patch that depend on this. Just to make sure we have it the right way.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - With those comments out of the way, the patch looks good to me. But I'd still like to see a follow up JIRA and a patch that depend on this. Just to make sure we have it the right way.
          Hide
          djp Junping Du added a comment -

          Thanks Vinod Kumar Vavilapalli for review and comments, I just deliver a quick demo patch in YARN-3586 (target for YARN-2928) to demonstrate how patch here helps to improve RM get back NM with limited number of collector addresses. Can you help to take a look?
          BTW, YARN-3212 for gracefully decommission need this patch too, so our target branch here is still for trunk/branch-2.

          Show
          djp Junping Du added a comment - Thanks Vinod Kumar Vavilapalli for review and comments, I just deliver a quick demo patch in YARN-3586 (target for YARN-2928 ) to demonstrate how patch here helps to improve RM get back NM with limited number of collector addresses. Can you help to take a look? BTW, YARN-3212 for gracefully decommission need this patch too, so our target branch here is still for trunk/branch-2.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 52s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 10m 3s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 111, now 112).
          -1 whitespace 0m 0s The patch has 16 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          -1 findbugs 2m 1s The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.
          -1 tools/hadoop tests 0m 20s Tests failed in hadoop-sls.
          -1 yarn tests 52m 30s Tests failed in hadoop-yarn-server-resourcemanager.
              91m 8s  



          Reason Tests
          FindBugs module:hadoop-sls
            Unwritten field:NodeInfo.java:[line 140]
          Failed unit tests hadoop.yarn.sls.nodemanager.TestNMSimulator
            hadoop.yarn.sls.TestSLSRunner
            hadoop.yarn.sls.appmaster.TestAMSimulator
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728844/YARN-3445-v2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6f62267
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/whitespace.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7807/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/7807/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 52s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 5s The applied patch generated 1 new checkstyle issues (total was 111, now 112). -1 whitespace 0m 0s The patch has 16 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. -1 findbugs 2m 1s The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. -1 tools/hadoop tests 0m 20s Tests failed in hadoop-sls. -1 yarn tests 52m 30s Tests failed in hadoop-yarn-server-resourcemanager.     91m 8s   Reason Tests FindBugs module:hadoop-sls   Unwritten field:NodeInfo.java: [line 140] Failed unit tests hadoop.yarn.sls.nodemanager.TestNMSimulator   hadoop.yarn.sls.TestSLSRunner   hadoop.yarn.sls.appmaster.TestAMSimulator   hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728844/YARN-3445-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6f62267 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/newPatchFindbugsWarningshadoop-sls.html hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7807/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7807/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7807/console This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Canceling the patch to address checkstyle / whitespace / findbugs / tests related complaints from Mr. Jenkins.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Canceling the patch to address checkstyle / whitespace / findbugs / tests related complaints from Mr. Jenkins.
          Hide
          djp Junping Du added a comment -

          Unit test failure should be unrelated. Fix checkstyle/whitespace/findbugs warnings in v3 patch.

          Show
          djp Junping Du added a comment - Unit test failure should be unrelated. Fix checkstyle/whitespace/findbugs warnings in v3 patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 13s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 javadoc 9m 53s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 6s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 57s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls.
          +1 yarn tests 52m 18s Tests passed in hadoop-yarn-server-resourcemanager.
              91m 44s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12734462/YARN-3445-v3.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / a5def58
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8042/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8042/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8042/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8042/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 13s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 9m 53s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 6s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 57s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls. +1 yarn tests 52m 18s Tests passed in hadoop-yarn-server-resourcemanager.     91m 44s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734462/YARN-3445-v3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a5def58 hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8042/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8042/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8042/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8042/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Vinod Kumar Vavilapalli, can you help to review it again? Thanks!

          Show
          djp Junping Du added a comment - Vinod Kumar Vavilapalli , can you help to review it again? Thanks!
          Hide
          djp Junping Du added a comment -

          Rebase the patch to trunk.

          Show
          djp Junping Du added a comment - Rebase the patch to trunk.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 19s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 36s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 5s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 14s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 53s Tests passed in hadoop-sls.
          +1 yarn tests 50m 48s Tests passed in hadoop-yarn-server-resourcemanager.
              92m 8s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12742445/YARN-3445-v3.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / aad6a7d
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8371/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8371/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8371/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8371/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 19s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 5s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 14s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 53s Tests passed in hadoop-sls. +1 yarn tests 50m 48s Tests passed in hadoop-yarn-server-resourcemanager.     92m 8s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12742445/YARN-3445-v3.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / aad6a7d hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8371/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8371/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8371/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8371/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Thanks Junping Du. Quick questions:

          • Regarding the extra memory this might take. Say you have 10K NM, 1000 apps per NM ( an application will remain in a RMNodeImpl list even though there are no more containers running on that NM, but the app still run on another NM.). That is around 10M entries. So it should be ok for RM.
          • Do you need synchronizedList in the following list? It looks like the access of runningApplications are protected by RMNodeImpl's readLock and writeLock.
            
            

            private final List<ApplicationId> runningApplications =
            Collections.synchronizedList(new ArrayList<ApplicationId>());

            {noromat}
          Show
          mingma Ming Ma added a comment - Thanks Junping Du . Quick questions: Regarding the extra memory this might take. Say you have 10K NM, 1000 apps per NM ( an application will remain in a RMNodeImpl list even though there are no more containers running on that NM, but the app still run on another NM.). That is around 10M entries. So it should be ok for RM. Do you need synchronizedList in the following list? It looks like the access of runningApplications are protected by RMNodeImpl's readLock and writeLock. private final List<ApplicationId> runningApplications = Collections.synchronizedList(new ArrayList<ApplicationId>()); {noromat}
          Hide
          djp Junping Du added a comment -

          Thanks for review and comments, Ming Ma!

          That is around 10M entries. So it should be ok for RM.

          ApplicationId only contains int (4 bytes) and long (8 bytes) field. Even consider java object header, padding and PB object overhead, should be far less than 100 bytes. Agree that it should be fine even in large scale as mentioned scenario.

          Do you need synchronizedList in the following list? It looks like the access of runningApplications are protected by RMNodeImpl's readLock and writeLock.

          Nice catch! Will replace synchronizedList will ArrayList and add some writeLocks (missing in previous patch).

          Show
          djp Junping Du added a comment - Thanks for review and comments, Ming Ma ! That is around 10M entries. So it should be ok for RM. ApplicationId only contains int (4 bytes) and long (8 bytes) field. Even consider java object header, padding and PB object overhead, should be far less than 100 bytes. Agree that it should be fine even in large scale as mentioned scenario. Do you need synchronizedList in the following list? It looks like the access of runningApplications are protected by RMNodeImpl's readLock and writeLock. Nice catch! Will replace synchronizedList will ArrayList and add some writeLocks (missing in previous patch).
          Hide
          djp Junping Du added a comment -

          Update patch in v4 patch according to Ming Ma's comments above.

          Show
          djp Junping Du added a comment - Update patch in v4 patch according to Ming Ma 's comments above.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 1s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 51s There were no new javac warning messages.
          +1 javadoc 9m 56s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 7s The applied patch generated 2 new checkstyle issues (total was 115, now 116).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 17s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls.
          +1 yarn tests 51m 4s Tests passed in hadoop-yarn-server-resourcemanager.
              92m 42s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743708/YARN-3445-v4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fc92d3e
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8437/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8437/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 1s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 9m 56s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 7s The applied patch generated 2 new checkstyle issues (total was 115, now 116). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 17s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls. +1 yarn tests 51m 4s Tests passed in hadoop-yarn-server-resourcemanager.     92m 42s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743708/YARN-3445-v4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fc92d3e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8437/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8437/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8437/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Fix check-style issue in v4.1 patch.

          Show
          djp Junping Du added a comment - Fix check-style issue in v4.1 patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 53s There were no new javac warning messages.
          +1 javadoc 10m 24s There were no new javadoc warning messages.
          +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 16s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 57s mvn install still works.
          +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
          +1 findbugs 2m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 57s Tests passed in hadoop-sls.
          -1 yarn tests 54m 46s Tests failed in hadoop-yarn-server-resourcemanager.
              98m 4s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743748/YARN-3445-v4.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fc92d3e
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8442/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8442/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8442/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8442/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 53s There were no new javac warning messages. +1 javadoc 10m 24s There were no new javadoc warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 16s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 2m 52s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 57s Tests passed in hadoop-sls. -1 yarn tests 54m 46s Tests failed in hadoop-yarn-server-resourcemanager.     98m 4s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743748/YARN-3445-v4.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fc92d3e hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8442/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8442/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8442/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8442/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 25s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 8m 42s There were no new javac warning messages.
          +1 javadoc 11m 26s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 10s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 57s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 2m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 55s Tests passed in hadoop-sls.
          +1 yarn tests 53m 32s Tests passed in hadoop-yarn-server-resourcemanager.
              98m 56s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743748/YARN-3445-v4.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / bf89ddb
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8445/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8445/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8445/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8445/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 25s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 42s There were no new javac warning messages. +1 javadoc 11m 26s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 10s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 2m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 55s Tests passed in hadoop-sls. +1 yarn tests 53m 32s Tests passed in hadoop-yarn-server-resourcemanager.     98m 56s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743748/YARN-3445-v4.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / bf89ddb hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8445/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8445/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8445/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8445/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Thanks Junping. Can you please check if it really needs to take write lock for the private methods addAppToRunningApps and removeAppFromRunningApps? It looks like RMNodeImpl's handle function already takes write lock for any state transition, and these private methods will only be invoked from state transition functions.

          Show
          mingma Ming Ma added a comment - Thanks Junping. Can you please check if it really needs to take write lock for the private methods addAppToRunningApps and removeAppFromRunningApps ? It looks like RMNodeImpl 's handle function already takes write lock for any state transition, and these private methods will only be invoked from state transition functions.
          Hide
          djp Junping Du added a comment -

          It looks like RMNodeImpl's handle function already takes write lock for any state transition, and these private methods will only be invoked from state transition functions.

          Nice catch, Ming Ma! I could miss the lock in the caller, so remove unnecessary write lock in v5 patch.

          Show
          djp Junping Du added a comment - It looks like RMNodeImpl's handle function already takes write lock for any state transition, and these private methods will only be invoked from state transition functions. Nice catch, Ming Ma ! I could miss the lock in the caller, so remove unnecessary write lock in v5 patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 16m 1s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 46s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 41s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls.
          -1 yarn tests 62m 54s Tests failed in hadoop-yarn-server-resourcemanager.
              102m 39s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743963/YARN-3445-v5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e0febce
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8448/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8448/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8448/testReport/
          Java 1.7.0_55
          uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8448/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 16m 1s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 41s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls. -1 yarn tests 62m 54s Tests failed in hadoop-yarn-server-resourcemanager.     102m 39s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743963/YARN-3445-v5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e0febce hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8448/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8448/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8448/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8448/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 6s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 javadoc 9m 53s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 6s The applied patch generated 1 new checkstyle issues (total was 115, now 115).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls.
          -1 yarn tests 48m 6s Tests failed in hadoop-yarn-server-resourcemanager.
              89m 41s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
            org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
            org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12743963/YARN-3445-v5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c9dd2ca
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8453/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8453/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 6s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 9m 53s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 6s The applied patch generated 1 new checkstyle issues (total was 115, now 115). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 51s Tests passed in hadoop-sls. -1 yarn tests 48m 6s Tests failed in hadoop-yarn-server-resourcemanager.     89m 41s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743963/YARN-3445-v5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c9dd2ca checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8453/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8453/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8453/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          The unit test failures seems not be related, and the same test failure get tracked by another JIRA YARN-3568. Fix minor check style issue in v5.1 patch.

          Show
          djp Junping Du added a comment - The unit test failures seems not be related, and the same test failure get tracked by another JIRA YARN-3568 . Fix minor check style issue in v5.1 patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 38s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 41s There were no new javac warning messages.
          +1 javadoc 9m 34s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 4s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 20s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 14s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls.
          +1 yarn tests 51m 2s Tests passed in hadoop-yarn-server-resourcemanager.
              91m 24s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12744509/YARN-3445-v5.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fffb15b
          hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8480/artifact/patchprocess/testrun_hadoop-sls.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8480/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8480/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8480/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 4s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 20s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 14s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 0m 52s Tests passed in hadoop-sls. +1 yarn tests 51m 2s Tests passed in hadoop-yarn-server-resourcemanager.     91m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12744509/YARN-3445-v5.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fffb15b hadoop-sls test log https://builds.apache.org/job/PreCommit-YARN-Build/8480/artifact/patchprocess/testrun_hadoop-sls.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/8480/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/8480/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/8480/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          +1 on the latest patch. Thanks Junping. I will wait until tomorrow to commit in case Vinod Kumar Vavilapalli or others have comments.

          Show
          mingma Ming Ma added a comment - +1 on the latest patch. Thanks Junping. I will wait until tomorrow to commit in case Vinod Kumar Vavilapalli or others have comments.
          Hide
          mingma Ming Ma added a comment -

          I have committed the patch to trunk and branch-2. Thanks Junping Du for the contribution. Thanks Vinod Kumar Vavilapalli and Jian He for the code review.

          Show
          mingma Ming Ma added a comment - I have committed the patch to trunk and branch-2. Thanks Junping Du for the contribution. Thanks Vinod Kumar Vavilapalli and Jian He for the code review.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8148 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8148/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8148 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8148/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #253 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/253/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #253 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/253/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #983 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/983/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #983 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/983/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #251 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/251/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #251 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/251/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2180/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2180 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2180/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-yarn-project/CHANGES.txt hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #241 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/241/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #241 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/241/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2199 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2199/)
          YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java
          • hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2199 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2199/ ) YARN-3445 . Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) (mingma: rev 08244264c0583472b9c4e16591cfde72c6db62a2) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMNodeTransitions.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/RMNodeWrapper.java hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/nodemanager/NodeInfo.java

            People

            • Assignee:
              djp Junping Du
              Reporter:
              djp Junping Du
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development