Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8911

ContainerScheduler incorrectly uses percentage number as the cpu resource utlization

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: nodemanager
    • Labels:
      None

      Description

      UPDATEper discussion below, the cpu resource utlization (ResourceUtilzation.cpu) is incorrectly documented as a percentage number in [0, 1.0] in the java doc. It is however, used as the number of vcores used throughout the system.  YARN-4597 is a victim of the  misleading java doc.

       

       

      ResourceUtilization represents the cpu utilization with a float number in [0, 1.0], i.e. the percentage of cpu usage across the node.  However, when Containers Monitor tracks the total aggregate resource utilization of all containers, it adds up the total number of vcores used by all running containers.

       

      (If you have 6 containers running, each consuming 1 vcore, you'd see the aggregated cpu container utilization being 6.0, but it's supposed to be always between 0 and 1.0)   See the code

        Attachments

        1. YARN-8911.00.patch
          1 kB
          Haibo Chen
        2. YARN-8911.01.patch
          10 kB
          Haibo Chen
        3. YARN-8911.02.patch
          22 kB
          Haibo Chen

          Issue Links

            Activity

              People

              • Assignee:
                haibochen Haibo Chen
                Reporter:
                haibochen Haibo Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: