Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4285

Display resource usage as percentage of queue and cluster in the RM UI

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, we display the memory and vcores allocated to an app in the RM UI. It would be useful to display the resources consumed as a %of the queue and the cluster to identify apps that are using a lot of resources.

      1. YARN-4285.001.patch
        9 kB
        Varun Vasudev
      2. YARN-4285.002.patch
        21 kB
        Varun Vasudev
      3. YARN-4285.003.patch
        25 kB
        Varun Vasudev
      4. YARN-4285.004.patch
        28 kB
        Varun Vasudev

        Activity

        Hide
        vvasudev Varun Vasudev added a comment -

        Uploaded a patch with an initial implementation.

        Show
        vvasudev Varun Vasudev added a comment - Uploaded a patch with an initial implementation.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 41s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 9m 39s There were no new javac warning messages.
        +1 javadoc 11m 56s There were no new javadoc warning messages.
        +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 59s The applied patch generated 13 new checkstyle issues (total was 21, now 34).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 57s mvn install still works.
        +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
        +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 64m 40s Tests passed in hadoop-yarn-server-resourcemanager.
            110m 59s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12767761/YARN-4285.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 0c4af0f
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9507/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9507/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 41s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 9m 39s There were no new javac warning messages. +1 javadoc 11m 56s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 59s The applied patch generated 13 new checkstyle issues (total was 21, now 34). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 64m 40s Tests passed in hadoop-yarn-server-resourcemanager.     110m 59s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767761/YARN-4285.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0c4af0f checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9507/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9507/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9507/console This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Varun Vasudev, thanks for working on this patch. This will be very useful!

        Some suggestions to the patch:

        • Instead of computing % in the web service component, it's better to compute % in the scheduler module, which we can consider resource calculator, etc. Now we have a place to do this, see org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.getResourceUsageReport(). After ApplicationResourceUsageReport is created, you can set % values to org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo.
        • And AppAttempt page can get AppInfo as well, may need to modify some interfaces.
        Show
        leftnoteasy Wangda Tan added a comment - Varun Vasudev , thanks for working on this patch. This will be very useful! Some suggestions to the patch: Instead of computing % in the web service component, it's better to compute % in the scheduler module, which we can consider resource calculator, etc. Now we have a place to do this, see org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt.getResourceUsageReport() . After ApplicationResourceUsageReport is created, you can set % values to org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo . And AppAttempt page can get AppInfo as well, may need to modify some interfaces.
        Hide
        vvasudev Varun Vasudev added a comment -

        Thanks for the feedback Wangda Tan. I've updated a new patch with your suggestions. Can you let me know if it looks ok? Thanks!

        Show
        vvasudev Varun Vasudev added a comment - Thanks for the feedback Wangda Tan . I've updated a new patch with your suggestions. Can you let me know if it looks ok? Thanks!
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks for updating, Varun Vasudev!

        Some comments:

        1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto.

        2)

            if (rmContext.getScheduler() instanceof YarnScheduler) {
              calc = rmContext.getScheduler().getResourceCalculator();
            }
        

        rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required.

        3) Is it better to change int percentage to float, for AppInfo and other APIs.

        4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?

        Show
        leftnoteasy Wangda Tan added a comment - Thanks for updating, Varun Vasudev ! Some comments: 1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto. 2) if (rmContext.getScheduler() instanceof YarnScheduler) { calc = rmContext.getScheduler().getResourceCalculator(); } rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required. 3) Is it better to change int percentage to float, for AppInfo and other APIs. 4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 38m 19s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
        +1 javac 11m 13s There were no new javac warning messages.
        +1 javadoc 13m 40s There were no new javadoc warning messages.
        +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 3m 13s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
        -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 2m 9s mvn install still works.
        +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse.
        -1 findbugs 9m 16s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 33s Tests passed in hadoop-yarn-api.
        -1 yarn tests 35m 28s Tests failed in hadoop-yarn-client.
        +1 yarn tests 2m 48s Tests passed in hadoop-yarn-common.
        +1 yarn tests 3m 59s Tests passed in hadoop-yarn-server-applicationhistoryservice.
        -1 yarn tests 267m 10s Tests failed in hadoop-yarn-server-resourcemanager.
            390m 20s  



        Reason Tests
        FindBugs module:hadoop-yarn-server-resourcemanager
        Failed unit tests hadoop.yarn.client.api.impl.TestYarnClient
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
          hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          hadoop.yarn.server.resourcemanager.TestClientRMService
          hadoop.yarn.server.resourcemanager.TestRM
          hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
          hadoop.yarn.server.resourcemanager.TestMoveApplication
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
          hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
          hadoop.yarn.server.resourcemanager.TestApplicationCleanup
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
          hadoop.yarn.server.resourcemanager.TestRMAdminService
          hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
          hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
          hadoop.yarn.server.resourcemanager.TestApplicationACLs
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
          hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
          hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
          hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
        Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient
          org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer
          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
          org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
          org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768082/YARN-4285.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 4c0bae2
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/whitespace.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-client.txt
        hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-common.txt
        hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9528/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9528/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 38m 19s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 11m 13s There were no new javac warning messages. +1 javadoc 13m 40s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 13s The applied patch generated 1 new checkstyle issues (total was 10, now 11). -1 whitespace 0m 2s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 2m 9s mvn install still works. +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse. -1 findbugs 9m 16s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 33s Tests passed in hadoop-yarn-api. -1 yarn tests 35m 28s Tests failed in hadoop-yarn-client. +1 yarn tests 2m 48s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 59s Tests passed in hadoop-yarn-server-applicationhistoryservice. -1 yarn tests 267m 10s Tests failed in hadoop-yarn-server-resourcemanager.     390m 20s   Reason Tests FindBugs module:hadoop-yarn-server-resourcemanager Failed unit tests hadoop.yarn.client.api.impl.TestYarnClient   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   hadoop.yarn.server.resourcemanager.TestMoveApplication   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestApplicationACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient   org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768082/YARN-4285.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c0bae2 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9528/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9528/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9528/console This message was automatically generated.
        Hide
        vvasudev Varun Vasudev added a comment -

        1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto.

        Fixed.

        2)

        if (rmContext.getScheduler() instanceof YarnScheduler)

        Unknown macro: { calc = rmContext.getScheduler().getResourceCalculator(); }

        rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required.

        Good catch! Fixed.

        3) Is it better to change int percentage to float, for AppInfo and other APIs.

        Fixed.

        4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly?

        Fixed.

        Show
        vvasudev Varun Vasudev added a comment - 1) Since ApplicationResourceUsageReport is public API, I suggest to rename parameter, getter/setter name from Perc to Percentage. Same as yarn_protos.proto. Fixed. 2) if (rmContext.getScheduler() instanceof YarnScheduler) Unknown macro: { calc = rmContext.getScheduler().getResourceCalculator(); } rmContext.getScheduler() should be always YarnScheduler, correct? This check maybe not required. Good catch! Fixed. 3) Is it better to change int percentage to float, for AppInfo and other APIs. Fixed. 4) Since this is not a trivial patch, could you add some tests to verify if SchedulerApplicationAttempt can return percentage properly? Fixed.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 26m 8s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 javac 9m 35s There were no new javac warning messages.
        +1 javadoc 10m 54s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 53s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
        +1 whitespace 0m 3s The patch has no lines that end in whitespace.
        +1 install 1m 39s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        +1 findbugs 6m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
        -1 yarn tests 34m 55s Tests failed in hadoop-yarn-client.
        +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common.
        +1 yarn tests 3m 51s Tests passed in hadoop-yarn-server-applicationhistoryservice.
        -1 yarn tests 327m 39s Tests failed in hadoop-yarn-server-resourcemanager.
            428m 42s  



        Reason Tests
        Failed unit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          hadoop.yarn.server.resourcemanager.TestRMHA
          hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
          hadoop.yarn.server.resourcemanager.TestApplicationACLs
          hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
          hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
          hadoop.yarn.server.resourcemanager.TestRM
          hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
          hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerApplicationAttempt
          hadoop.yarn.server.resourcemanager.TestClientRMService
          hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
        Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
          org.apache.hadoop.yarn.client.api.impl.TestNMClient
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
          org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
          org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer
          org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
          org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
          org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
          org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768239/YARN-4285.003.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 124a412
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-client.txt
        hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-common.txt
        hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9543/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9543/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 26m 8s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 9m 35s There were no new javac warning messages. +1 javadoc 10m 54s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 53s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 3s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 6m 50s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. -1 yarn tests 34m 55s Tests failed in hadoop-yarn-client. +1 yarn tests 2m 1s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 51s Tests passed in hadoop-yarn-server-applicationhistoryservice. -1 yarn tests 327m 39s Tests failed in hadoop-yarn-server-resourcemanager.     428m 42s   Reason Tests Failed unit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestApplicationACLs   hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate   hadoop.yarn.server.resourcemanager.TestRM   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerApplicationAttempt   hadoop.yarn.server.resourcemanager.TestClientRMService   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager Timed out tests org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup   org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer   org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768239/YARN-4285.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 124a412 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9543/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9543/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9543/console This message was automatically generated.
        Hide
        vvasudev Varun Vasudev added a comment -

        The failing tests are due to a deadlock caused by calling getQueueInfo within the getResourceUsageReport() function in SchedulerApplicationAttempt. After speaking with Wangda Tan offline, he suggested that there's no need for the getQueueInfo call to be synchronized.

        Uploaded a new patch with the fix.

        Show
        vvasudev Varun Vasudev added a comment - The failing tests are due to a deadlock caused by calling getQueueInfo within the getResourceUsageReport() function in SchedulerApplicationAttempt. After speaking with Wangda Tan offline, he suggested that there's no need for the getQueueInfo call to be synchronized. Uploaded a new patch with the fix.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 29m 49s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 javac 11m 27s There were no new javac warning messages.
        +1 javadoc 12m 17s There were no new javadoc warning messages.
        +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 3m 56s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
        +1 whitespace 0m 4s The patch has no lines that end in whitespace.
        +1 install 1m 49s mvn install still works.
        +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
        +1 findbugs 8m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
        +1 yarn tests 7m 11s Tests passed in hadoop-yarn-client.
        +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common.
        -1 yarn tests 3m 7s Tests failed in hadoop-yarn-server-applicationhistoryservice.
        +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common.
        -1 yarn tests 0m 19s Tests failed in hadoop-yarn-server-resourcemanager.
            83m 32s  



        Reason Tests
        Failed unit tests hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore
        Failed build hadoop-yarn-server-resourcemanager



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 600ad7b
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-client.txt
        hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-common.txt
        hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
        hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9552/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9552/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 29m 49s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 11m 27s There were no new javac warning messages. +1 javadoc 12m 17s There were no new javadoc warning messages. +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 3m 56s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 4s The patch has no lines that end in whitespace. +1 install 1m 49s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 8m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 7m 11s Tests passed in hadoop-yarn-client. +1 yarn tests 2m 3s Tests passed in hadoop-yarn-common. -1 yarn tests 3m 7s Tests failed in hadoop-yarn-server-applicationhistoryservice. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-server-common. -1 yarn tests 0m 19s Tests failed in hadoop-yarn-server-resourcemanager.     83m 32s   Reason Tests Failed unit tests hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore Failed build hadoop-yarn-server-resourcemanager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 600ad7b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9552/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9552/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9552/console This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Tried this patch locally, looks great! Also tried to sort applications based on queue percentage, cluster percentage, all work as expected.
        Will commit shortly if no objections.

        Show
        leftnoteasy Wangda Tan added a comment - Tried this patch locally, looks great! Also tried to sort applications based on queue percentage, cluster percentage, all work as expected. Will commit shortly if no objections.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 22m 51s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 javac 7m 59s There were no new javac warning messages.
        +1 javadoc 10m 22s There were no new javadoc warning messages.
        +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 45s The applied patch generated 1 new checkstyle issues (total was 10, now 11).
        +1 whitespace 0m 5s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 7m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api.
        +1 yarn tests 7m 3s Tests passed in hadoop-yarn-client.
        +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common.
        +1 yarn tests 3m 19s Tests passed in hadoop-yarn-server-applicationhistoryservice.
        +1 yarn tests 0m 26s Tests passed in hadoop-yarn-server-common.
        -1 yarn tests 55m 48s Tests failed in hadoop-yarn-server-resourcemanager.
            124m 30s  



        Reason Tests
        Timed out tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 15eb84b
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-client.txt
        hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-common.txt
        hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt
        hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9554/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9554/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 51s Pre-patch trunk has 3 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 22s There were no new javadoc warning messages. +1 release audit 0m 30s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 45s The applied patch generated 1 new checkstyle issues (total was 10, now 11). +1 whitespace 0m 5s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 7m 39s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 0m 24s Tests passed in hadoop-yarn-api. +1 yarn tests 7m 3s Tests passed in hadoop-yarn-client. +1 yarn tests 2m 2s Tests passed in hadoop-yarn-common. +1 yarn tests 3m 19s Tests passed in hadoop-yarn-server-applicationhistoryservice. +1 yarn tests 0m 26s Tests passed in hadoop-yarn-server-common. -1 yarn tests 55m 48s Tests failed in hadoop-yarn-server-resourcemanager.     124m 30s   Reason Tests Timed out tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768364/YARN-4285.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 15eb84b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/trunkFindbugsWarningshadoop-yarn-common.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-client test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-client.txt hadoop-yarn-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-common.txt hadoop-yarn-server-applicationhistoryservice test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-applicationhistoryservice.txt hadoop-yarn-server-common test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-common.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9554/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9554/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9554/console This message was automatically generated.
        Hide
        vvasudev Varun Vasudev added a comment -

        The test failures are not related to the patch. They pass on my machine.

        Show
        vvasudev Varun Vasudev added a comment - The test failures are not related to the patch. They pass on my machine.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to trunk/branch-2, thanks Varun Vasudev!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2, thanks Varun Vasudev !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8710 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8710/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1323 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1323/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #599 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/599/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2477/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2530 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2530/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #540 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/540/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/)
        YARN-4285. Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #588 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/588/ ) YARN-4285 . Display resource usage as percentage of queue and cluster in (wangda: rev 3cc73773eb26f7469c99b25a76814d6fad0be28e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationResourceUsageReport.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryManagerOnTimelineStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationResourceUsageReportPBImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Hide
        jianhe Jian He added a comment -

        Varun Vasudev, Wangda Tan, looks like the % of queue/cluster for the app does not include reserved resources. However, the queue's used resource in the UI does include reserved resource too. This caused the total amount of resources for all apps in the table does not equal to the total used resource shown in the queue.
        Should we consider reserved resource too in the % of queue/cluster for an app ?

        Show
        jianhe Jian He added a comment - Varun Vasudev , Wangda Tan , looks like the % of queue/cluster for the app does not include reserved resources. However, the queue's used resource in the UI does include reserved resource too. This caused the total amount of resources for all apps in the table does not equal to the total used resource shown in the queue. Should we consider reserved resource too in the % of queue/cluster for an app ?
        Hide
        sunilg Sunil G added a comment -

        Hi Jian He, in YARN-4678, we were trying to take reserved capacity out of used capacity in UI. And given fox for same. Could you also pls take a look.

        Show
        sunilg Sunil G added a comment - Hi Jian He , in YARN-4678 , we were trying to take reserved capacity out of used capacity in UI. And given fox for same. Could you also pls take a look.
        Hide
        vvasudev Varun Vasudev added a comment -

        Jian He - it makes sense to remove reserved resources from the used resources, but do we know why we counted reserved resources as part of used resources in the first place?

        Show
        vvasudev Varun Vasudev added a comment - Jian He - it makes sense to remove reserved resources from the used resources, but do we know why we counted reserved resources as part of used resources in the first place?
        Hide
        leftnoteasy Wangda Tan added a comment -

        Jian He,

        However, the queue's used resource in the UI does include reserved resource too.

        IIUC, it should queue's used resource should include reserved resources

        Varun Vasudev,

        it makes sense to remove reserved resources from the used resources,

        Actually I think we should include reserved resources by used resources, unless we can show them together on UI. See my #2 comment.

        ...but do we know why we counted reserved resources as part of used resources in the first place?

        The reason is, if we create a reserved container under a queue, we need to make sure it doesn't go beyond queue's max capacity. In another word, if resource is reserved by someone, nobody else can use that part of resources.
        From YARN's perspective, a queue has 99G allocated (not reserved) + 1G reserved is as same as 1G allocated + 99G reserved. To be more transparent to users and avoid answer questions like: "why my total allocated resource is always less than total resources, used resource should be allocated + reserved.

        Show
        leftnoteasy Wangda Tan added a comment - Jian He , However, the queue's used resource in the UI does include reserved resource too. IIUC, it should queue's used resource should include reserved resources Varun Vasudev , it makes sense to remove reserved resources from the used resources, Actually I think we should include reserved resources by used resources, unless we can show them together on UI. See my #2 comment . ...but do we know why we counted reserved resources as part of used resources in the first place? The reason is, if we create a reserved container under a queue, we need to make sure it doesn't go beyond queue's max capacity. In another word, if resource is reserved by someone, nobody else can use that part of resources. From YARN's perspective, a queue has 99G allocated (not reserved) + 1G reserved is as same as 1G allocated + 99G reserved. To be more transparent to users and avoid answer questions like: "why my total allocated resource is always less than total resources, used resource should be allocated + reserved.
        Hide
        vvasudev Varun Vasudev added a comment -

        Thanks for the explanation Wangda Tan. So it looks like for internal accounting purposes we should count reserved containers as used resources but from a web UI perspective, we should subtract reserved resources and show them seperately?

        Show
        vvasudev Varun Vasudev added a comment - Thanks for the explanation Wangda Tan . So it looks like for internal accounting purposes we should count reserved containers as used resources but from a web UI perspective, we should subtract reserved resources and show them seperately?
        Hide
        sunilg Sunil G added a comment -

        Yes Wangda Tan and Varun Vasudev
        It will be good if we can show them separately as we already showing this resources separately in cluster metrics UI. Also Wangda's comment looks fine, which s to show 2 resource in 2 colors in same queue bar. I making some changes to try this. I will update.

        Show
        sunilg Sunil G added a comment - Yes Wangda Tan and Varun Vasudev It will be good if we can show them separately as we already showing this resources separately in cluster metrics UI. Also Wangda's comment looks fine, which s to show 2 resource in 2 colors in same queue bar. I making some changes to try this. I will update.

          People

          • Assignee:
            vvasudev Varun Vasudev
            Reporter:
            vvasudev Varun Vasudev
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development