Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-799

CgroupsLCEResourcesHandler tries to write to cgroup.procs


    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.4-alpha, 2.0.5-alpha
    • Fix Version/s: 2.1.0-beta
    • Component/s: nodemanager
    • Labels:


      The implementation of


      Tells the container-executor to write PIDs to cgroup.procs:

        public String getResourcesOption(ContainerId containerId) {
          String containerName = containerId.toString();
          StringBuilder sb = new StringBuilder("cgroups=");
          if (isCpuWeightEnabled()) {
            sb.append(pathForCgroup(CONTROLLER_CPU, containerName) + "/cgroup.procs");
          if (sb.charAt(sb.length() - 1) == ',') {
            sb.deleteCharAt(sb.length() - 1);
          return sb.toString();

      Apparently, this file has not always been writeable:


      The RHEL version of the Linux kernel that I'm using has a CGroup module that has a non-writeable cgroup.procs file.

      $ uname -a
      Linux criccomi-ld 2.6.32-131.4.1.el6.x86_64 #1 SMP Fri Jun 10 10:54:26 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

      As a result, when the container-executor tries to run, it fails with this error message:

      fprintf(LOGFILE, "Failed to write pid %s (%d) to file %s - %s\n",

      This is because the executor is given a resource by the CgroupsLCEResourcesHandler that includes cgroup.procs, which is non-writeable:

      $ pwd
      $ ls -l
      total 0
      rrr- 1 criccomi eng 0 Jun 11 14:43 cgroup.procs
      rw-rr- 1 criccomi eng 0 Jun 11 14:43 cpu.rt_period_us
      rw-rr- 1 criccomi eng 0 Jun 11 14:43 cpu.rt_runtime_us
      rw-rr- 1 criccomi eng 0 Jun 11 14:43 cpu.shares
      rw-rr- 1 criccomi eng 0 Jun 11 14:43 notify_on_release
      rw-rr- 1 criccomi eng 0 Jun 11 14:43 tasks

      I patched CgroupsLCEResourcesHandler to use /tasks instead of /cgroup.procs, and this appears to have fixed the problem.

      I can think of several potential resolutions to this ticket:

      1. Ignore the problem, and make people patch YARN when they hit this issue.
      2. Write to /tasks instead of /cgroup.procs for everyone
      3. Check permissioning on /cgroup.procs prior to writing to it, and fall back to /tasks.
      4. Add a config to yarn-site that lets admins specify which file to write to.



        1. YARN-799.0.patch
          0.9 kB
          Chris Riccomini



            • Assignee:
              criccomini Chris Riccomini
              criccomini Chris Riccomini
            • Votes:
              0 Vote for this issue
              13 Start watching this issue


              • Created: