Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9045

Container's cpu usage limit should be soft with a relaxation factor

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.6.0, 3.1.1
    • Fix Version/s: None
    • Component/s: nodemanager
    • Labels:
      None
    • Flags:
      Patch

      Description

      There are two modes in yarn container cpu limitation. The default is Shared mode which guarantees the requested resource and can also use other spare CPU if available. Another one is Strict mode, container's resource is limited to allocated resource strictly.

      If we want to limit the resource usage of the container and can use a certain percentage of spare CPU if available, the above two modes cannot satisfy our demand. We can define a relax coefficient(yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage.ratio) to relax the restrictions on resources in strict mode.

       
      Attachments are different modes comparison. Especifically, the last picture is the maximum percentage of container using CPU when the relaxation coefficient is 1.5.

        Attachments

        1. shared_mode.png
          590 kB
          Zhaohui Xin
        2. strict_mode.png
          116 kB
          Zhaohui Xin
        3. strict_relax_mode.png
          131 kB
          Zhaohui Xin
        4. YARN-9045.001.patch
          3 kB
          Zhaohui Xin
        5. YARN-9045.002.patch
          5 kB
          Zhaohui Xin

          Activity

            People

            • Assignee:
              uranus Zhaohui Xin
              Reporter:
              uranus Zhaohui Xin
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: