Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.5.0
    • Labels:
      None

      Description

      Post YARN-2 we need to add cpu information to queue metrics.

      1. YARN-322.patch
        48 kB
        Nathan Roberts
      2. YARN-322.patch
        39 kB
        Nathan Roberts

        Issue Links

          Activity

          Hide
          Timothy St. Clair added a comment -

          I'm still on my learning curve, but from what I have seen there is limited information that is passed by the Resource. Given some of the other JIRA's around memory, wouldn't it make sense to expand the scope to enable some type of NVP set for Node attributes, with cpu capabilities being part of that set.

          One could envision the scope expanding to include GPUs, interconnect capabilities, etc, as not all machines are =.

          This discrimination will allow ApplicationManagers to better filter desired resources for their problem.

          Also feel free to tell me I'm wrong, as I'm still learning .

          Show
          Timothy St. Clair added a comment - I'm still on my learning curve, but from what I have seen there is limited information that is passed by the Resource. Given some of the other JIRA's around memory, wouldn't it make sense to expand the scope to enable some type of NVP set for Node attributes, with cpu capabilities being part of that set. One could envision the scope expanding to include GPUs, interconnect capabilities, etc, as not all machines are =. This discrimination will allow ApplicationManagers to better filter desired resources for their problem. Also feel free to tell me I'm wrong, as I'm still learning .
          Hide
          Nathan Roberts added a comment -

          Patch adds vcore information to QueueMetrics, UI and webservices. Can break into separate jiras if that's more desirable.

          Show
          Nathan Roberts added a comment - Patch adds vcore information to QueueMetrics, UI and webservices. Can break into separate jiras if that's more desirable.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12643759/YARN-322.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestNodesPage

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3711//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3711//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643759/YARN-322.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices org.apache.hadoop.yarn.server.resourcemanager.webapp.TestNodesPage +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3711//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3711//console This message is automatically generated.
          Hide
          Nathan Roberts added a comment -

          Missed a few tests. new patch.

          Show
          Nathan Roberts added a comment - Missed a few tests. new patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12643831/YARN-322.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3713//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3713//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643831/YARN-322.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3713//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3713//console This message is automatically generated.
          Show
          Nathan Roberts added a comment - TestRMRestart failure isn't related to this change and is being tracked via YARN-1906 . https://issues.apache.org/jira/browse/YARN-1906?focusedCommentId=13989883&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13989883
          Hide
          Nathan Roberts added a comment -

          Arun, does this patch address what you were looking for? Happy to adjust if not.

          Show
          Nathan Roberts added a comment - Arun, does this patch address what you were looking for? Happy to adjust if not.
          Hide
          Nathan Roberts added a comment -

          virtual cores was added to queue metrics via YARN-598. So resolving this jira as duplicate.

          I created YARN-2072 to address adding vcores to UI and webservices.

          Feel free to re-open if this doesn't address everything,

          Show
          Nathan Roberts added a comment - virtual cores was added to queue metrics via YARN-598 . So resolving this jira as duplicate. I created YARN-2072 to address adding vcores to UI and webservices. Feel free to re-open if this doesn't address everything,

            People

            • Assignee:
              Nathan Roberts
              Reporter:
              Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development