Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      Post YARN-2 we need to add cpu information to queue metrics.

      1. YARN-322.patch
        39 kB
        Nathan Roberts
      2. YARN-322.patch
        48 kB
        Nathan Roberts

        Issue Links

          Activity

          Karthik Kambatla (Inactive) made changes -
          Fix Version/s 2.5.0 [ 12326262 ]
          Nathan Roberts made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Hide
          Nathan Roberts added a comment -

          virtual cores was added to queue metrics via YARN-598. So resolving this jira as duplicate.

          I created YARN-2072 to address adding vcores to UI and webservices.

          Feel free to re-open if this doesn't address everything,

          Show
          Nathan Roberts added a comment - virtual cores was added to queue metrics via YARN-598 . So resolving this jira as duplicate. I created YARN-2072 to address adding vcores to UI and webservices. Feel free to re-open if this doesn't address everything,
          Hide
          Nathan Roberts added a comment -

          Arun, does this patch address what you were looking for? Happy to adjust if not.

          Show
          Nathan Roberts added a comment - Arun, does this patch address what you were looking for? Happy to adjust if not.
          Show
          Nathan Roberts added a comment - TestRMRestart failure isn't related to this change and is being tracked via YARN-1906 . https://issues.apache.org/jira/browse/YARN-1906?focusedCommentId=13989883&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13989883
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12643831/YARN-322.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3713//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3713//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643831/YARN-322.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3713//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3713//console This message is automatically generated.
          Nathan Roberts made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Nathan Roberts added a comment -

          Missed a few tests. new patch.

          Show
          Nathan Roberts added a comment - Missed a few tests. new patch.
          Nathan Roberts made changes -
          Attachment YARN-322.patch [ 12643831 ]
          Nathan Roberts made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12643759/YARN-322.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestNodesPage

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3711//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3711//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12643759/YARN-322.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices org.apache.hadoop.yarn.server.resourcemanager.webapp.TestNodesPage +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3711//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3711//console This message is automatically generated.
          Nathan Roberts made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 2.4.0 [ 12326142 ]
          Target Version/s 3.0.0, 2.5.0 [ 12323268, 12326262 ]
          Assignee Arun C Murthy [ acmurthy ] Nathan Roberts [ nroberts ]
          Nathan Roberts made changes -
          Attachment YARN-322.patch [ 12643759 ]
          Hide
          Nathan Roberts added a comment -

          Patch adds vcore information to QueueMetrics, UI and webservices. Can break into separate jiras if that's more desirable.

          Show
          Nathan Roberts added a comment - Patch adds vcore information to QueueMetrics, UI and webservices. Can break into separate jiras if that's more desirable.
          Arun C Murthy made changes -
          Fix Version/s 2.5.0 [ 12326262 ]
          Fix Version/s 2.4.0 [ 12326142 ]
          Arun C Murthy made changes -
          Fix Version/s 2.4.0 [ 12326142 ]
          Fix Version/s 2.3.0 [ 12325256 ]
          Arun C Murthy made changes -
          Fix Version/s 2.3.0 [ 12325256 ]
          Fix Version/s 2.4.0 [ 12324589 ]
          Arun C Murthy made changes -
          Fix Version/s 2.3.0 [ 12324589 ]
          Fix Version/s 2.1.0-beta [ 12324029 ]
          Hide
          Timothy St. Clair added a comment -

          I'm still on my learning curve, but from what I have seen there is limited information that is passed by the Resource. Given some of the other JIRA's around memory, wouldn't it make sense to expand the scope to enable some type of NVP set for Node attributes, with cpu capabilities being part of that set.

          One could envision the scope expanding to include GPUs, interconnect capabilities, etc, as not all machines are =.

          This discrimination will allow ApplicationManagers to better filter desired resources for their problem.

          Also feel free to tell me I'm wrong, as I'm still learning .

          Show
          Timothy St. Clair added a comment - I'm still on my learning curve, but from what I have seen there is limited information that is passed by the Resource. Given some of the other JIRA's around memory, wouldn't it make sense to expand the scope to enable some type of NVP set for Node attributes, with cpu capabilities being part of that set. One could envision the scope expanding to include GPUs, interconnect capabilities, etc, as not all machines are =. This discrimination will allow ApplicationManagers to better filter desired resources for their problem. Also feel free to tell me I'm wrong, as I'm still learning .
          Arun C Murthy made changes -
          Fix Version/s 2.0.4-beta [ 12324029 ]
          Fix Version/s 2.0.3-alpha [ 12323272 ]
          Junping Du made changes -
          Link This issue is related to YARN-347 [ YARN-347 ]
          Arun C Murthy made changes -
          Component/s capacityscheduler [ 12319324 ]
          Component/s scheduler [ 12319325 ]
          Arun C Murthy made changes -
          Field Original Value New Value
          Fix Version/s 2.0.3-alpha [ 12323272 ]
          Arun C Murthy created issue -

            People

            • Assignee:
              Nathan Roberts
              Reporter:
              Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development