Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4995

FairScheduler: Display per-queue demand on the scheduler page

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For now there is no demand resource information for queues on the scheduler page.

      Just using used resource information, it is hard for us to judge whether the queue is needy (demand > used , but cluster has no available resource). And without demand resource information, modifying min/max resource for queue is not accurate.

      1. demo_screenshot.png
        139 kB
        xupeng
      2. YARN-4995.001.patch
        3 kB
        xupeng
      3. YARN-4995.002.patch
        3 kB
        xupeng

        Issue Links

          Activity

          Hide
          xupener xupeng added a comment -

          Attach patch.

          Display demand resource for queues on the scheduler page. And users can get demand resource information through yarn web service interface.

          Show
          xupener xupeng added a comment - Attach patch. Display demand resource for queues on the scheduler page. And users can get demand resource information through yarn web service interface.
          Hide
          xupener xupeng added a comment -

          add demo screenshot

          Show
          xupener xupeng added a comment - add demo screenshot
          Hide
          xupener xupeng added a comment -

          Hi Junping Du

          could you please review this patch and issue ? FYI , why can't i assign this issue to myself ?

          Show
          xupener xupeng added a comment - Hi Junping Du could you please review this patch and issue ? FYI , why can't i assign this issue to myself ?
          Hide
          djp Junping Du added a comment -

          Hi xupeng, I just add you to YARN contributor so you can assign issue to yourself now.
          Your patch is about Fair Scheduler. I assume Karthik Kambatla will review it?

          Show
          djp Junping Du added a comment - Hi xupeng , I just add you to YARN contributor so you can assign issue to yourself now. Your patch is about Fair Scheduler. I assume Karthik Kambatla will review it?
          Hide
          xupener xupeng added a comment -

          Hi Junping Du:

          Thanks a lot for your reply.

          Show
          xupener xupeng added a comment - Hi Junping Du: Thanks a lot for your reply.
          Hide
          xupener xupeng added a comment -

          Hi Karthik Kambatla

          could you please review this patch and issue ?

          Show
          xupener xupeng added a comment - Hi Karthik Kambatla could you please review this patch and issue ?
          Hide
          kasha Karthik Kambatla added a comment -

          +1, checking this in.

          Show
          kasha Karthik Kambatla added a comment - +1, checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Actually, let us wait until we get a Jenkins run in.

          Show
          kasha Karthik Kambatla added a comment - Actually, let us wait until we get a Jenkins run in.
          Hide
          xupener xupeng added a comment -

          hi Karthik Kambatla

          Thanks for your reply.

          I don't know why the former patch has not triggered Jenkins run. Is it because i am not in the contributor list when i upload the patch file?

          And i resubmitted the patch file with a new version num.

          Show
          xupener xupeng added a comment - hi Karthik Kambatla : Thanks for your reply. I don't know why the former patch has not triggered Jenkins run. Is it because i am not in the contributor list when i upload the patch file? And i resubmitted the patch file with a new version num.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 31s trunk passed
          +1 compile 0m 26s trunk passed with JDK v1.8.0_91
          +1 compile 0m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 5s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.8.0_91
          +1 javac 0m 24s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_95
          +1 javac 0m 27s the patch passed
          +1 checkstyle 0m 16s the patch passed
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 19s the patch passed
          +1 javadoc 0m 20s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95
          -1 unit 44m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          -1 unit 44m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          105m 38s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801907/YARN-4995.002.patch
          JIRA Issue YARN-4995
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 62873450c24c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c1cc6ac
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11316/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11316/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 31s trunk passed +1 compile 0m 26s trunk passed with JDK v1.8.0_91 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 5s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 27s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 30s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_91 +1 javac 0m 24s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 19s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_95 -1 unit 44m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 44m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 105m 38s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_91 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801907/YARN-4995.002.patch JIRA Issue YARN-4995 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 62873450c24c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c1cc6ac Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11316/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11316/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11316/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 44s trunk passed
          +1 compile 0m 39s trunk passed with JDK v1.8.0_92
          +1 compile 0m 33s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 20s trunk passed
          +1 mvnsite 0m 38s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 11s trunk passed
          +1 javadoc 0m 30s trunk passed with JDK v1.8.0_92
          +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 32s the patch passed
          +1 compile 0m 34s the patch passed with JDK v1.8.0_92
          +1 javac 0m 34s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.7.0_95
          +1 javac 0m 31s the patch passed
          +1 checkstyle 0m 17s the patch passed
          +1 mvnsite 0m 37s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 22s the patch passed
          +1 javadoc 0m 26s the patch passed with JDK v1.8.0_92
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_95
          -1 unit 51m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92.
          -1 unit 51m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 28s Patch does not generate ASF License warnings.
          122m 46s



          Reason Tests
          JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801911/YARN-4995.002.patch
          JIRA Issue YARN-4995
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 148ac05f124c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c1cc6ac
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11317/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11317/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 44s trunk passed +1 compile 0m 39s trunk passed with JDK v1.8.0_92 +1 compile 0m 33s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 38s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 11s trunk passed +1 javadoc 0m 30s trunk passed with JDK v1.8.0_92 +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 32s the patch passed +1 compile 0m 34s the patch passed with JDK v1.8.0_92 +1 javac 0m 34s the patch passed +1 compile 0m 31s the patch passed with JDK v1.7.0_95 +1 javac 0m 31s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 0m 37s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 22s the patch passed +1 javadoc 0m 26s the patch passed with JDK v1.8.0_92 +1 javadoc 0m 26s the patch passed with JDK v1.7.0_95 -1 unit 51m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92. -1 unit 51m 38s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 122m 46s Reason Tests JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801911/YARN-4995.002.patch JIRA Issue YARN-4995 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 148ac05f124c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c1cc6ac Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11317/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11317/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11317/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          xupener xupeng added a comment -

          hi Karthik Kambatla

          Got jenkins output, and these two errors are not generated by this patch.

          Show
          xupener xupeng added a comment - hi Karthik Kambatla : Got jenkins output, and these two errors are not generated by this patch.
          Hide
          kasha Karthik Kambatla added a comment -

          Cool. Looks good per my previous review. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - Cool. Looks good per my previous review. Checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for the contribution, xupeng. Just committed this to trunk, branch-2, and branch-2.8.

          Show
          kasha Karthik Kambatla added a comment - Thanks for the contribution, xupeng. Just committed this to trunk, branch-2, and branch-2.8.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9749 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9749/)
          YARN-4995. FairScheduler: Display per-queue demand on the scheduler (kasha: rev 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/FairSchedulerPage.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/FairSchedulerQueueInfo.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9749 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9749/ ) YARN-4995 . FairScheduler: Display per-queue demand on the scheduler (kasha: rev 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/FairSchedulerPage.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/FairSchedulerQueueInfo.java
          Hide
          xupener xupeng added a comment -

          Hi Karthik Kambatla

          Thanks for the review

          Show
          xupener xupeng added a comment - Hi Karthik Kambatla Thanks for the review
          Hide
          imstefanlee stefanlee added a comment -

          xupeng thanks your jira, i have a question,the demand resource you mentioned will beyond the max resource a lot,IMO,the demand resource is used resource plus request resource,because of node locality and rack locality ,a task's request resource may include many containers,e.g. <20,"node1","memory:1G,cpu:1",1,true> ,<20,"node2","memory:1G,cpu:1",1,true> ,<20,"rack1","memory:1G,cpu:1",1,true> ,<20,"rack2","memory:1G,cpu:1",1,true> ,<20,"","memory:1G,cpu:1",1,true> , a MR application may has 20 maps and need 20<memory: 1G,cpu:1>,but the demand resource will reach 5*20*<memory:1G,cpu:1> ,so the demand resource cant help user to modify a queue's min/max resource clearly. I dont know my option correct or not?

          Show
          imstefanlee stefanlee added a comment - xupeng thanks your jira, i have a question,the demand resource you mentioned will beyond the max resource a lot,IMO,the demand resource is used resource plus request resource,because of node locality and rack locality ,a task's request resource may include many containers,e.g. <20,"node1","memory:1G,cpu:1",1,true> ,<20,"node2","memory:1G,cpu:1",1,true> ,<20,"rack1","memory:1G,cpu:1",1,true> ,<20,"rack2","memory:1G,cpu:1",1,true> ,<20," ","memory:1G,cpu:1",1,true> , a MR application may has 20 maps and need 20 <memory: 1G,cpu:1>,but the demand resource will reach 5*20*<memory:1G,cpu:1> ,so the demand resource cant help user to modify a queue's min/max resource clearly. I dont know my option correct or not?

            People

            • Assignee:
              xupener xupeng
              Reporter:
              xupener xupeng
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development