Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6384

Add configuration property to increase max CPU usage when strict-resource-usage is true with cgroups

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      When using cgroups on yarn, if yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage is false, user may get very more cpu time than expected based on the vcores. There should be a upper limit even resource-usage is not strict, just like a percentage which user can get more than promised by vcores. I think it's important in a shared cluster.

      Attachments

        1. YARN-6384-5.patch
          7 kB
          dengkai
        2. YARN-6384-4.patch
          7 kB
          dengkai
        3. YARN-6384-3.patch
          6 kB
          dengkai
        4. YARN-6384-2.patch
          10 kB
          dengkai
        5. YARN-6384-1.patch
          10 kB
          dengkai
        6. YARN-6384-0.patch
          6 kB
          dengkai

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-5936 when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              lolee_k dengkai
              lolee_k dengkai
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: