Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4285

Display resource usage as percentage of queue and cluster in the RM UI

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, we display the memory and vcores allocated to an app in the RM UI. It would be useful to display the resources consumed as a %of the queue and the cluster to identify apps that are using a lot of resources.

      1. YARN-4285.001.patch
        9 kB
        Varun Vasudev
      2. YARN-4285.002.patch
        21 kB
        Varun Vasudev
      3. YARN-4285.003.patch
        25 kB
        Varun Vasudev
      4. YARN-4285.004.patch
        28 kB
        Varun Vasudev

        Issue Links

          Activity

          Hide
          vvasudev Varun Vasudev added a comment -

          Uploaded a patch with an initial implementation.

          Show
          vvasudev Varun Vasudev added a comment - Uploaded a patch with an initial implementation.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 41s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 9m 39s There were no new javac warning messages.
          +1 javadoc 11m 56s There were no new javadoc warning messages.
          +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 59s The applied patch generated 13 new checkstyle issues (total was 21, now 34).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 57s mvn install still works.
          +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
          +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 64m 40s Tests passed in hadoop-yarn-server-resourcemanager.
              110m 59s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767761/YARN-4285.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0c4af0f
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9507/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9507/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 41s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 39s There were no new javac warning messages. +1 javadoc 11m 56s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 59s The applied patch generated 13 new checkstyle issues (total was 21, now 34). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 64m 40s Tests passed in hadoop-yarn-server-resourcemanager.     110m 59s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767761/YARN-4285.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0c4af0f checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9507/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9507/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Varun Vasudev, thanks for working on this patch. This will be very useful!

          Some suggestions to the patch:

          • Instead of computing % in the web service component, it's better to compute % in the scheduler module, which we can consider resource calculator, etc. Now we have a place to do this, see org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.getResourceUsageReport(). After ApplicationResourceUsageReport is created, you can set % values to org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo.
          • And AppAttempt page can get AppInfo as well, may need to modify some interfaces.
          Show
          leftnoteasy Wangda Tan added a comment - Varun Vasudev , thanks for working on this patch. This will be very useful! Some suggestions to the patch: Instead of computing % in the web service component, it's better to compute % in the scheduler module, which we can consider resource calculator, etc. Now we have a place to do this, see org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.getResourceUsageReport() . After ApplicationResourceUsageReport is created, you can set % values to org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo . And AppAttempt page can get AppInfo as well, may need to modify some interfaces.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for the feedback Wangda Tan. I've updated a new patch with your suggestions. Can you let me know if it looks ok? Thanks!

          Show
          vvasudev Varun Vasudev added a comment - Thanks for the feedback Wangda Tan . I've updated a new patch with your suggestions. Can you let me know if it looks ok? Thanks!
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for updating, Varun Vasudev!

          Some comments:

          1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto.

          2)

              if (rmContext.getScheduler() instanceof YarnScheduler) {
                calc = rmContext.getScheduler().getResourceCalculator();
              }
          

          rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required.

          3) Is it better to change int percentage to float, for AppInfo and other APIs.

          4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for updating, Varun Vasudev ! Some comments: 1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto. 2) if (rmContext.getScheduler() instanceof YarnScheduler) { calc = rmContext.getScheduler().getResourceCalculator(); } rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required. 3) Is it better to change int percentage to float, for AppInfo and other APIs. 4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 38m 19s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
          +1 javac 11m 13s There were no new javac warning messages.
          +1 javadoc 13m 40s There were no new javadoc warning messages.
          +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 13s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
          -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 2m 9s mvn install still works.
          +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse.
          -1 findbugs 9m 16s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 33s Tests passed in hadoop-yarn-api.
          -1 yarn tests 35m 28s Tests failed in hadoop-yarn-client.
          +1 yarn tests 2m 48s Tests passed in hadoop-yarn-common.
          +1 yarn tests 3m 59s Tests passed in hadoop-yarn-server-applicationhistoryservice.
          -1 yarn tests 267m 10s Tests failed in hadoop-yarn-server-resourcemanager.
              390m 20s  



          Reason Tests
          FindBugs module:hadoop-yarn-server-resourcemanager
          Failed unit tests hadoop.yarn.client.api.impl.TestYarnClient
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestClientRMService
            hadoop.yarn.server.resourcemanager.TestRM
            hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
            hadoop.yarn.server.resourcemanager.TestMoveApplication
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
            hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestApplicationACLs
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
            hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
            hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
          Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
            org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer
            org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
            org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768082/YARN-4285.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4c0bae2
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/whitespace.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-client.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9528/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9528/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 38m 19s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 11m 13s There were no new javac warning messages. +1 javadoc 13m 40s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 13s The applied patch generated 1 new checkstyle issues (total was 10, now 11). -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 2m 9s mvn install still works. +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse. -1 findbugs 9m 16s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 33s Tests passed in hadoop-yarn-api. -1 yarn tests 35m 28s Tests failed in hadoop-yarn-client. +1 yarn tests 2m 48s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 59s Tests passed in hadoop-yarn-server-applicationhistoryservice. -1 yarn tests 267m 10s Tests failed in hadoop-yarn-server-resourcemanager.     390m 20s   Reason Tests FindBugs module:hadoop-yarn-server-resourcemanager Failed unit tests hadoop.yarn.client.api.impl.TestYarnClient   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   hadoop.yarn.server.resourcemanager.TestMoveApplication   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestApplicationACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient   org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768082/YARN-4285.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c0bae2 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9528/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9528/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto.

          Fixed.

          2)

          if (rmContext.getScheduler() instanceof YarnScheduler)

          Unknown macro: { calc = rmContext.getScheduler().getResourceCalculator(); }

          rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required.

          Good catch! Fixed.

          3) Is it better to change int percentage to float, for AppInfo and other APIs.

          Fixed.

          4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?

          Fixed.

          Show
          vvasudev Varun Vasudev added a comment - 1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto. Fixed. 2) if (rmContext.getScheduler() instanceof YarnScheduler) Unknown macro: { calc = rmContext.getScheduler().getResourceCalculator(); } rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required. Good catch! Fixed. 3) Is it better to change int percentage to float, for AppInfo and other APIs. Fixed. 4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly? Fixed.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 26m 8s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 9m 35s There were no new javac warning messages.
          +1 javadoc 10m 54s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 53s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
          +1 whitespace 0m 3s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 6m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          -1 yarn tests 34m 55s Tests failed in hadoop-yarn-client.
          +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
          +1 yarn tests 3m 51s Tests passed in hadoop-yarn-server-applicationhistoryservice.
          -1 yarn tests 327m 39s Tests failed in hadoop-yarn-server-resourcemanager.
              428m 42s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
            hadoop.yarn.server.resourcemanager.TestApplicationACLs
            hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
            hadoop.yarn.server.resourcemanager.TestRM
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerApplicationAttempt
            hadoop.yarn.server.resourcemanager.TestClientRMService
            hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
          Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
            org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
            org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer
            org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
            org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
            org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
            org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768239/YARN-4285.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 124a412
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-client.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9543/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9543/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 26m 8s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 9m 35s There were no new javac warning messages. +1 javadoc 10m 54s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 53s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 6m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. -1 yarn tests 34m 55s Tests failed in hadoop-yarn-client. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 51s Tests passed in hadoop-yarn-server-applicationhistoryservice. -1 yarn tests 327m 39s Tests failed in hadoop-yarn-server-resourcemanager.     428m 42s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestApplicationACLs   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerApplicationAttempt   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup   org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer   org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768239/YARN-4285.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 124a412 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9543/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9543/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          The failing tests are due to a deadlock caused by calling getQueueInfo within the getResourceUsageReport() function in SchedulerApplicationAttempt. After speaking with Wangda Tan offline, he suggested that there's no need for the getQueueInfo call to be synchronized.

          Uploaded a new patch with the fix.

          Show
          vvasudev Varun Vasudev added a comment - The failing tests are due to a deadlock caused by calling getQueueInfo within the getResourceUsageReport() function in SchedulerApplicationAttempt. After speaking with Wangda Tan offline, he suggested that there's no need for the getQueueInfo call to be synchronized. Uploaded a new patch with the fix.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 29m 49s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 11m 27s There were no new javac warning messages.
          +1 javadoc 12m 17s There were no new javadoc warning messages.
          +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 3m 56s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
          +1 whitespace 0m 4s The patch has no lines that end in whitespace.
          +1 install 1m 49s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 8m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 7m 11s Tests passed in hadoop-yarn-client.
          +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common.
          -1 yarn tests 3m 7s Tests failed in hadoop-yarn-server-applicationhistoryservice.
          +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common.
          -1 yarn tests 0m 19s Tests failed in hadoop-yarn-server-resourcemanager.
              83m 32s  



          Reason Tests
          Failed unit tests hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore
          Failed build hadoop-yarn-server-resourcemanager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 600ad7b
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-client.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9552/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9552/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 29m 49s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 11m 27s There were no new javac warning messages. +1 javadoc 12m 17s There were no new javadoc warning messages. +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 56s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 4s The patch has no lines that end in whitespace. +1 install 1m 49s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 8m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 7m 11s Tests passed in hadoop-yarn-client. +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common. -1 yarn tests 3m 7s Tests failed in hadoop-yarn-server-applicationhistoryservice. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common. -1 yarn tests 0m 19s Tests failed in hadoop-yarn-server-resourcemanager.     83m 32s   Reason Tests Failed unit tests hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore Failed build hadoop-yarn-server-resourcemanager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 600ad7b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9552/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9552/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Tried this patch locally, looks great! Also tried to sort applications based on queue percentage, cluster percentage, all work as expected.
          Will commit shortly if no objections.

          Show
          leftnoteasy Wangda Tan added a comment - Tried this patch locally, looks great! Also tried to sort applications based on queue percentage, cluster percentage, all work as expected. Will commit shortly if no objections.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 22m 51s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 7m 59s There were no new javac warning messages.
          +1 javadoc 10m 22s There were no new javadoc warning messages.
          +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 2m 45s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
          +1 whitespace 0m 5s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 7m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
          +1 yarn tests 7m 3s Tests passed in hadoop-yarn-client.
          +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common.
          +1 yarn tests 3m 19s Tests passed in hadoop-yarn-server-applicationhistoryservice.
          +1 yarn tests 0m 26s Tests passed in hadoop-yarn-server-common.
          -1 yarn tests 55m 48s Tests failed in hadoop-yarn-server-resourcemanager.
              124m 30s  



          Reason Tests
          Timed out tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 15eb84b
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
          hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-api.txt
          hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-client.txt
          hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-common.txt
          hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
          hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
          hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9554/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9554/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 51s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 22s There were no new javadoc warning messages. +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 45s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 5s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 7m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 7m 3s Tests passed in hadoop-yarn-client. +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 19s Tests passed in hadoop-yarn-server-applicationhistoryservice. +1 yarn tests 0m 26s Tests passed in hadoop-yarn-server-common. -1 yarn tests 55m 48s Tests failed in hadoop-yarn-server-resourcemanager.     124m 30s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 15eb84b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9554/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9554/console This message was automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          The test failures are not related to the patch. They pass on my machine.

          Show
          vvasudev Varun Vasudev added a comment - The test failures are not related to the patch. They pass on my machine.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to trunk/branch-2, thanks Varun Vasudev!

          Show
          leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2, thanks Varun Vasudev !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/)
          YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
          Hide
          jianhe Jian He added a comment -

          Varun Vasudev, Wangda Tan, looks like the % of queue/cluster for the app does not include reserved resources. However, the queue's used resource in the UI does include reserved resource too. This caused the total amount of resources for all apps in the table does not equal to the total used resource shown in the queue.
          Should we consider reserved resource too in the % of queue/cluster for an app ?

          Show
          jianhe Jian He added a comment - Varun Vasudev , Wangda Tan , looks like the % of queue/cluster for the app does not include reserved resources. However, the queue's used resource in the UI does include reserved resource too. This caused the total amount of resources for all apps in the table does not equal to the total used resource shown in the queue. Should we consider reserved resource too in the % of queue/cluster for an app ?
          Hide
          sunilg Sunil G added a comment -

          Hi Jian He, in YARN-4678, we were trying to take reserved capacity out of used capacity in UI. And given fox for same. Could you also pls take a look.

          Show
          sunilg Sunil G added a comment - Hi Jian He , in YARN-4678 , we were trying to take reserved capacity out of used capacity in UI. And given fox for same. Could you also pls take a look.
          Hide
          vvasudev Varun Vasudev added a comment -

          Jian He - it makes sense to remove reserved resources from the used resources, but do we know why we counted reserved resources as part of used resources in the first place?

          Show
          vvasudev Varun Vasudev added a comment - Jian He - it makes sense to remove reserved resources from the used resources, but do we know why we counted reserved resources as part of used resources in the first place?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Jian He,

          However, the queue's used resource in the UI does include reserved resource too.

          IIUC, it should queue's used resource should include reserved resources

          Varun Vasudev,

          it makes sense to remove reserved resources from the used resources,

          Actually I think we should include reserved resources by used resources, unless we can show them together on UI. See my #2 comment.

          ...but do we know why we counted reserved resources as part of used resources in the first place?

          The reason is, if we create a reserved container under a queue, we need to make sure it doesn't go beyond queue's max capacity. In another word, if resource is reserved by someone, nobody else can use that part of resources.
          From YARN's perspective, a queue has 99G allocated (not reserved) + 1G reserved is as same as 1G allocated + 99G reserved. To be more transparent to users and avoid answer questions like: "why my total allocated resource is always less than total resources, used resource should be allocated + reserved.

          Show
          leftnoteasy Wangda Tan added a comment - Jian He , However, the queue's used resource in the UI does include reserved resource too. IIUC, it should queue's used resource should include reserved resources Varun Vasudev , it makes sense to remove reserved resources from the used resources, Actually I think we should include reserved resources by used resources, unless we can show them together on UI. See my #2 comment . ...but do we know why we counted reserved resources as part of used resources in the first place? The reason is, if we create a reserved container under a queue, we need to make sure it doesn't go beyond queue's max capacity. In another word, if resource is reserved by someone, nobody else can use that part of resources. From YARN's perspective, a queue has 99G allocated (not reserved) + 1G reserved is as same as 1G allocated + 99G reserved. To be more transparent to users and avoid answer questions like: "why my total allocated resource is always less than total resources, used resource should be allocated + reserved.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for the explanation Wangda Tan. So it looks like for internal accounting purposes we should count reserved containers as used resources but from a web UI perspective, we should subtract reserved resources and show them seperately?

          Show
          vvasudev Varun Vasudev added a comment - Thanks for the explanation Wangda Tan . So it looks like for internal accounting purposes we should count reserved containers as used resources but from a web UI perspective, we should subtract reserved resources and show them seperately?
          Hide
          sunilg Sunil G added a comment -

          Yes Wangda Tan and Varun Vasudev
          It will be good if we can show them separately as we already showing this resources separately in cluster metrics UI. Also Wangda's comment looks fine, which s to show 2 resource in 2 colors in same queue bar. I making some changes to try this. I will update.

          Show
          sunilg Sunil G added a comment - Yes Wangda Tan and Varun Vasudev It will be good if we can show them separately as we already showing this resources separately in cluster metrics UI. Also Wangda's comment looks fine, which s to show 2 resource in 2 colors in same queue bar. I making some changes to try this. I will update.

            People

            • Assignee:
              vvasudev Varun Vasudev
              Reporter:
              vvasudev Varun Vasudev
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development