Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5849

Automatically create YARN control group for pre-mounted cgroups

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Yarn can be launched with linux-container-executor.cgroups.mount set to false. It will search for the cgroup mount paths set up by the administrator parsing the /etc/mtab file. You can also specify resource.percentage-physical-cpu-limit to limit the CPU resources assigned to containers.
      linux-container-executor.cgroups.hierarchy is the root of the settings of all YARN containers. If this is specified but not created YARN will fail at startup:
      Caused by: java.io.FileNotFoundException: /cgroups/cpu/hadoop-yarn/cpu.cfs_period_us (Permission denied)
      org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler.updateCgroup(CgroupsLCEResourcesHandler.java:263)

      This JIRA is about automatically creating YARN control group in the case above. It reduces the cost of administration.

        Attachments

        1. YARN-5849.000.patch
          14 kB
          Miklos Szegedi
        2. YARN-5849.001.patch
          14 kB
          Miklos Szegedi
        3. YARN-5849.002.patch
          34 kB
          Miklos Szegedi
        4. YARN-5849.003.patch
          39 kB
          Miklos Szegedi
        5. YARN-5849.004.patch
          41 kB
          Miklos Szegedi
        6. YARN-5849.005.patch
          41 kB
          Miklos Szegedi
        7. YARN-5849.006.patch
          41 kB
          Miklos Szegedi
        8. YARN-5849.007.patch
          43 kB
          Miklos Szegedi
        9. YARN-5849.008.patch
          43 kB
          Miklos Szegedi

          Activity

            People

            • Assignee:
              miklos.szegedi@cloudera.com Miklos Szegedi
              Reporter:
              miklos.szegedi@cloudera.com Miklos Szegedi
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: