Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5849

Automatically create YARN control group for pre-mounted cgroups

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.3, 3.0.0-alpha1, 3.0.0-alpha2
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Yarn can be launched with linux-container-executor.cgroups.mount set to false. It will search for the cgroup mount paths set up by the administrator parsing the /etc/mtab file. You can also specify resource.percentage-physical-cpu-limit to limit the CPU resources assigned to containers.
      linux-container-executor.cgroups.hierarchy is the root of the settings of all YARN containers. If this is specified but not created YARN will fail at startup:
      Caused by: java.io.FileNotFoundException: /cgroups/cpu/hadoop-yarn/cpu.cfs_period_us (Permission denied)
      org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler.updateCgroup(CgroupsLCEResourcesHandler.java:263)

      This JIRA is about automatically creating YARN control group in the case above. It reduces the cost of administration.

      1. YARN-5849.000.patch
        14 kB
        Miklos Szegedi
      2. YARN-5849.001.patch
        14 kB
        Miklos Szegedi
      3. YARN-5849.002.patch
        34 kB
        Miklos Szegedi
      4. YARN-5849.003.patch
        39 kB
        Miklos Szegedi
      5. YARN-5849.004.patch
        41 kB
        Miklos Szegedi
      6. YARN-5849.005.patch
        41 kB
        Miklos Szegedi
      7. YARN-5849.006.patch
        41 kB
        Miklos Szegedi
      8. YARN-5849.007.patch
        43 kB
        Miklos Szegedi
      9. YARN-5849.008.patch
        43 kB
        Miklos Szegedi

        Activity

        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        The patch creates the yarn control group, if it does not exists. I also added some unit tests that verify the edge cases.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - The patch creates the yarn control group, if it does not exists. I also added some unit tests that verify the edge cases.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 7m 11s trunk passed
        +1 compile 8m 43s trunk passed
        +1 checkstyle 0m 51s trunk passed
        +1 mvnsite 1m 8s trunk passed
        +1 mvneclipse 0m 50s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 49s trunk passed
        +1 javadoc 0m 47s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 33s the patch passed
        +1 compile 4m 57s the patch passed
        +1 javac 4m 57s the patch passed
        -0 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 2 unchanged - 1 fixed = 3 total (was 3)
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 39s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 58s the patch passed
        +1 javadoc 0m 46s the patch passed
        -1 unit 15m 56s hadoop-yarn-server-nodemanager in the patch failed.
        +1 unit 0m 19s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 38s The patch does not generate ASF License warnings.
        55m 44s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837852/YARN-5849.000.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5daabd3bd855 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / de3b4aa
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13814/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13814/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13814/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13814/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 11s trunk passed +1 compile 8m 43s trunk passed +1 checkstyle 0m 51s trunk passed +1 mvnsite 1m 8s trunk passed +1 mvneclipse 0m 50s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 49s trunk passed +1 javadoc 0m 47s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 33s the patch passed +1 compile 4m 57s the patch passed +1 javac 4m 57s the patch passed -0 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 2 unchanged - 1 fixed = 3 total (was 3) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 39s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 58s the patch passed +1 javadoc 0m 46s the patch passed -1 unit 15m 56s hadoop-yarn-server-nodemanager in the patch failed. +1 unit 0m 19s hadoop-yarn-site in the patch passed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 55m 44s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837852/YARN-5849.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5daabd3bd855 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / de3b4aa Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13814/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13814/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13814/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13814/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Fixing checkstyle issue.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Fixing checkstyle issue.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 29s Maven dependency ordering for branch
        +1 mvninstall 7m 15s trunk passed
        +1 compile 6m 55s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 44s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 54s trunk passed
        +1 javadoc 0m 48s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 33s the patch passed
        +1 compile 4m 58s the patch passed
        +1 javac 4m 58s the patch passed
        +1 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
        +1 mvnsite 1m 1s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 59s the patch passed
        +1 javadoc 0m 47s the patch passed
        +1 unit 16m 9s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 21s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 43s The patch does not generate ASF License warnings.
        54m 34s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837868/YARN-5849.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 882019f46a81 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 3dbad5d
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13815/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13815/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 7m 15s trunk passed +1 compile 6m 55s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 44s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 54s trunk passed +1 javadoc 0m 48s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 33s the patch passed +1 compile 4m 58s the patch passed +1 javac 4m 58s the patch passed +1 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3) +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 59s the patch passed +1 javadoc 0m 47s the patch passed +1 unit 16m 9s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 21s hadoop-yarn-site in the patch passed. +1 asflicense 0m 43s The patch does not generate ASF License warnings. 54m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837868/YARN-5849.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 882019f46a81 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3dbad5d Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13815/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13815/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Thank you Miklos Szegedi for patch

        Patch helps in debugging with detailed logging.

        1. As per current implementation we do we support 'deferred' mounting of specific controllers.But when enable mount is false we try to create cGroupPrefix for all CGroupController.
        2. if (!FileUtil.canWrite(yarnHierarchy)) check should be in else case rt?
        Show
        bibinchundatt Bibin A Chundatt added a comment - Thank you Miklos Szegedi for patch Patch helps in debugging with detailed logging. As per current implementation we do we support 'deferred' mounting of specific controllers.But when enable mount is false we try to create cGroupPrefix for all CGroupController . if (!FileUtil.canWrite(yarnHierarchy)) check should be in else case rt?
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you for the comment Bibin A Chundatt!

        1. The patch applies to the scenario, when enable mount is false. The group is created in other cases. The current implementation throws an exception if any controller does not have the group created and writable. What is your more specific concern?
        2. If I put the line into the else branch, we would miss the ret.put call in case the group was just created. I could put ret.put inside both branches, but that would make the code more complicated I think. Does this answer your concern?

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you for the comment Bibin A Chundatt ! 1. The patch applies to the scenario, when enable mount is false. The group is created in other cases. The current implementation throws an exception if any controller does not have the group created and writable. What is your more specific concern? 2. If I put the line into the else branch, we would miss the ret.put call in case the group was just created. I could put ret.put inside both branches, but that would make the code more complicated I think. Does this answer your concern?
        Hide
        bibinchundatt Bibin A Chundatt added a comment - - edited

        1. The patch applies to the scenario, when enable mount is false. The group is created in other cases. The current implementation throws an exception if any controller does not have the group created and writable. What is your more specific concern?

        IIUC even if the mount patch exists, its not required to create yarnHierarchy for all CGroupController now we can individually configure in ResourceHandlerModule which resource chain to use. For example as per current implementation even if yarn is not going to use memory monitor will create yarnHierarchy folder .Which is not required to be created.

        Show
        bibinchundatt Bibin A Chundatt added a comment - - edited 1. The patch applies to the scenario, when enable mount is false. The group is created in other cases. The current implementation throws an exception if any controller does not have the group created and writable. What is your more specific concern? IIUC even if the mount patch exists, its not required to create yarnHierarchy for all CGroupController now we can individually configure in ResourceHandlerModule which resource chain to use. For example as per current implementation even if yarn is not going to use memory monitor will create yarnHierarchy folder .Which is not required to be created.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        I see now what you mean. The way the code behaves without mount before my patch is the following. When we start initializing the resource chain, we look for the yarn cgroup in every controller and throw an exception and exit if the yarn group does not exist or not writable, regardless of whether it is enabled or not.

          public enum CGroupController {
            CPU("cpu"),
            NET_CLS("net_cls"),
            BLKIO("blkio"),
            MEMORY("memory");
        ...  
          }
        ...
              for (CGroupController controller : CGroupController.values()) {
                String name = controller.getName();
                String controllerPath = findControllerInMtab(name, parsedMtab);
        
                if (controllerPath != null) {
                  File f = new File(controllerPath + "/" + cGroupPrefix);
        
                  if (FileUtil.canWrite(f)) {
                    ret.put(controller, controllerPath);
                  } else {
                    String error =
                        new StringBuffer("Mount point Based on mtab file: ")
                            .append(mtab)
                            .append(". Controller mount point not writable for: ")
                            .append(name).toString();
        
                    LOG.error(error);
                    throw new ResourceHandlerException(error);
                  }
                } else {
                  LOG.warn("Controller not mounted but automount disabled: " + name);
                }
              }
        

        I will update my patch to avoid this and do the check and creation only, if the subsystem in question is enabled.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - I see now what you mean. The way the code behaves without mount before my patch is the following. When we start initializing the resource chain, we look for the yarn cgroup in every controller and throw an exception and exit if the yarn group does not exist or not writable, regardless of whether it is enabled or not. public enum CGroupController { CPU( "cpu" ), NET_CLS( "net_cls" ), BLKIO( "blkio" ), MEMORY( "memory" ); ... } ... for (CGroupController controller : CGroupController.values()) { String name = controller.getName(); String controllerPath = findControllerInMtab(name, parsedMtab); if (controllerPath != null ) { File f = new File(controllerPath + "/" + cGroupPrefix); if (FileUtil.canWrite(f)) { ret.put(controller, controllerPath); } else { String error = new StringBuffer ( "Mount point Based on mtab file: " ) .append(mtab) .append( ". Controller mount point not writable for : " ) .append(name).toString(); LOG.error(error); throw new ResourceHandlerException(error); } } else { LOG.warn( "Controller not mounted but automount disabled: " + name); } } I will update my patch to avoid this and do the check and creation only, if the subsystem in question is enabled.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        I updated the patch with a better implementation based on your comments. CGroupsHandlerImpl used to have a single mountCGroupController function to handle the case when mount is enabled. With this change there will be a function called initializeCGroupController. It will call initializePreMountedCGroupController to make sure the group exist in the pre-mounted directories. Otherwise, it calls mountCGroupController, if we are dealing with a pre-mounted controller. All resource handlers are supposed to call initializeCGroupController as they do with mountCGroupController now. This way initializeCGroupController is called once only if the subsystem in question is enabled ensuring that we do not touch the cgroup mount points of external subsystems.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - I updated the patch with a better implementation based on your comments. CGroupsHandlerImpl used to have a single mountCGroupController function to handle the case when mount is enabled. With this change there will be a function called initializeCGroupController. It will call initializePreMountedCGroupController to make sure the group exist in the pre-mounted directories. Otherwise, it calls mountCGroupController, if we are dealing with a pre-mounted controller. All resource handlers are supposed to call initializeCGroupController as they do with mountCGroupController now. This way initializeCGroupController is called once only if the subsystem in question is enabled ensuring that we do not touch the cgroup mount points of external subsystems.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 17s Maven dependency ordering for branch
        +1 mvninstall 7m 49s trunk passed
        +1 compile 9m 27s trunk passed
        +1 checkstyle 0m 58s trunk passed
        +1 mvnsite 1m 4s trunk passed
        +1 mvneclipse 0m 43s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 53s trunk passed
        +1 javadoc 0m 48s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 0m 32s the patch passed
        +1 compile 5m 3s the patch passed
        +1 javac 5m 3s the patch passed
        -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 45 unchanged - 4 fixed = 47 total (was 49)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 35s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 1m 8s the patch passed
        +1 javadoc 0m 40s the patch passed
        +1 unit 16m 25s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 20s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 47s The patch does not generate ASF License warnings.
        57m 43s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838243/YARN-5849.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 7649cbd59489 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 59ee8b7
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13846/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13846/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13846/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 7m 49s trunk passed +1 compile 9m 27s trunk passed +1 checkstyle 0m 58s trunk passed +1 mvnsite 1m 4s trunk passed +1 mvneclipse 0m 43s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 53s trunk passed +1 javadoc 0m 48s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 32s the patch passed +1 compile 5m 3s the patch passed +1 javac 5m 3s the patch passed -0 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 45 unchanged - 4 fixed = 47 total (was 49) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 8s the patch passed +1 javadoc 0m 40s the patch passed +1 unit 16m 25s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 20s hadoop-yarn-site in the patch passed. +1 asflicense 0m 47s The patch does not generate ASF License warnings. 57m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838243/YARN-5849.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7649cbd59489 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 59ee8b7 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13846/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13846/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13846/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Fix checkstyle issues

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Fix checkstyle issues
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 7m 2s trunk passed
        +1 compile 6m 17s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 50s trunk passed
        +1 javadoc 0m 46s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 31s the patch passed
        +1 compile 4m 45s the patch passed
        +1 javac 4m 45s the patch passed
        +1 checkstyle 0m 47s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 25 unchanged - 23 fixed = 25 total (was 48)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 40s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 0m 57s the patch passed
        +1 javadoc 0m 27s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 231 unchanged - 5 fixed = 231 total (was 236)
        +1 javadoc 0m 18s hadoop-yarn-site in the patch passed.
        +1 unit 14m 39s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 19s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 39s The patch does not generate ASF License warnings.
        51m 20s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838279/YARN-5849.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux aa42f1828b05 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 71adf44
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13849/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13849/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 2s trunk passed +1 compile 6m 17s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 42s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 50s trunk passed +1 javadoc 0m 46s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 31s the patch passed +1 compile 4m 45s the patch passed +1 javac 4m 45s the patch passed +1 checkstyle 0m 47s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 25 unchanged - 23 fixed = 25 total (was 48) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 0m 57s the patch passed +1 javadoc 0m 27s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 231 unchanged - 5 fixed = 231 total (was 236) +1 javadoc 0m 18s hadoop-yarn-site in the patch passed. +1 unit 14m 39s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 19s hadoop-yarn-site in the patch passed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 51m 20s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838279/YARN-5849.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux aa42f1828b05 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 71adf44 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13849/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13849/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Bibin A Chundatt, does the latest change address your concerns?

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Bibin A Chundatt , does the latest change address your concerns?
        Hide
        templedf Daniel Templeton added a comment -

        While we wait for Bibin A Chundatt to review the latest patch, I have a few minor comments:

        • Thanks for fixing the javadocs!
        • In CGroupsHandler, let's leave the MTAB_FILE constant there for use in the new constructor.
        • The new constructor needs javadocs.
        • Looks to me like getErrorWithDetails() should just return the String.
        • In getErrorWithDetails() you may want to use UserGroupInformation.getCurrentUser() instead of the system property.
        • Missing a @param in the javadoc for testPreMountedControllerInitialization()
        • It would be good to have messages in all your asserts.
        • In the docs, it would be good to be clearer about the new behavior, e.g. that the mount point will be created if it doesn't exist, that it needs write permissions, etc. Basically a brain dump to help the poor admin who's trying to debug a broken cluster.
        • Does yarn-defaults.xml also need to be updated?
        Show
        templedf Daniel Templeton added a comment - While we wait for Bibin A Chundatt to review the latest patch, I have a few minor comments: Thanks for fixing the javadocs! In CGroupsHandler , let's leave the MTAB_FILE constant there for use in the new constructor. The new constructor needs javadocs. Looks to me like getErrorWithDetails() should just return the String . In getErrorWithDetails() you may want to use UserGroupInformation.getCurrentUser() instead of the system property. Missing a @param in the javadoc for testPreMountedControllerInitialization() It would be good to have messages in all your asserts. In the docs, it would be good to be clearer about the new behavior, e.g. that the mount point will be created if it doesn't exist, that it needs write permissions, etc. Basically a brain dump to help the poor admin who's trying to debug a broken cluster. Does yarn-defaults.xml also need to be updated?
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you for the review, Daniel Templeton! I addressed all the issues that you raised except the one below and I will send the update soon.
        "In getErrorWithDetails() you may want to use UserGroupInformation.getCurrentUser() instead of the system property."
        I need to show the system user that will access the cgroup folder. I think System.getProperty("user.name") is the right one here based on my research. UserGroupInformation.getCurrentUser().getRealUser() returns null when running from the tests. Since there are other users of System.getProperty("user.name") in the Hadoop code base, I suggest using it here too.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you for the review, Daniel Templeton ! I addressed all the issues that you raised except the one below and I will send the update soon. "In getErrorWithDetails() you may want to use UserGroupInformation.getCurrentUser() instead of the system property." I need to show the system user that will access the cgroup folder. I think System.getProperty("user.name") is the right one here based on my research. UserGroupInformation.getCurrentUser().getRealUser() returns null when running from the tests. Since there are other users of System.getProperty("user.name") in the Hadoop code base, I suggest using it here too.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 6m 47s trunk passed
        +1 compile 5m 6s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 28s trunk passed
        +1 mvneclipse 0m 52s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 1m 11s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 0m 58s the patch passed
        +1 compile 4m 39s the patch passed
        +1 javac 4m 39s the patch passed
        +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49)
        +1 mvnsite 1m 26s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 2m 1s the patch passed
        +1 javadoc 0m 31s hadoop-yarn-common in the patch passed.
        +1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235)
        +1 javadoc 0m 14s hadoop-yarn-site in the patch passed.
        +1 unit 2m 26s hadoop-yarn-common in the patch passed.
        +1 unit 13m 41s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 14s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 29s The patch does not generate ASF License warnings.
        55m 16s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840719/YARN-5849.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
        uname Linux 0d4ea8781c60 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a2b1ff0
        Default Java 1.8.0_111
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14082/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14082/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14082/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 47s trunk passed +1 compile 5m 6s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 28s trunk passed +1 mvneclipse 0m 52s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 46s trunk passed +1 javadoc 1m 11s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 0m 58s the patch passed +1 compile 4m 39s the patch passed +1 javac 4m 39s the patch passed +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49) +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 49s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 1s the patch passed +1 javadoc 0m 31s hadoop-yarn-common in the patch passed. +1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235) +1 javadoc 0m 14s hadoop-yarn-site in the patch passed. +1 unit 2m 26s hadoop-yarn-common in the patch passed. +1 unit 13m 41s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 14s hadoop-yarn-site in the patch passed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 55m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840719/YARN-5849.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 0d4ea8781c60 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a2b1ff0 Default Java 1.8.0_111 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14082/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14082/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14082/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Fixing whitespace issue

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Fixing whitespace issue
        Hide
        templedf Daniel Templeton added a comment -

        Thanks for the update. A couple doc issues:

        • In yarn-defaults.xml, you have "then the he." Probably not what you meant. Also, the sentence with that phrase doesn't make sense to me. Do you mean that if the property is false, the directory doesn't exist, and the user has write permission, then the directory will be created?
        • Same for NodeManagerCgroups.md
        Show
        templedf Daniel Templeton added a comment - Thanks for the update. A couple doc issues: In yarn-defaults.xml , you have "then the he." Probably not what you meant. Also, the sentence with that phrase doesn't make sense to me. Do you mean that if the property is false, the directory doesn't exist, and the user has write permission, then the directory will be created? Same for NodeManagerCgroups.md
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Addressing comments

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Addressing comments
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you Daniel Templeton. I fixed the issue.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you Daniel Templeton . I fixed the issue.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 13s Maven dependency ordering for branch
        +1 mvninstall 7m 32s trunk passed
        +1 compile 5m 10s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 54s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 1m 52s trunk passed
        +1 javadoc 1m 14s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 1m 0s the patch passed
        +1 compile 4m 46s the patch passed
        +1 javac 4m 46s the patch passed
        +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49)
        +1 mvnsite 1m 26s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 2m 9s the patch passed
        +1 javadoc 0m 32s hadoop-yarn-common in the patch passed.
        +1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235)
        +1 javadoc 0m 14s hadoop-yarn-site in the patch passed.
        +1 unit 2m 28s hadoop-yarn-common in the patch passed.
        +1 unit 14m 8s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 13s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 31s The patch does not generate ASF License warnings.
        57m 12s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841120/YARN-5849.006.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
        uname Linux 3e78ee482e1c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4fca94f
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14129/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14129/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 7m 32s trunk passed +1 compile 5m 10s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 54s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 52s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 0s the patch passed +1 compile 4m 46s the patch passed +1 javac 4m 46s the patch passed +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49) +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 9s the patch passed +1 javadoc 0m 32s hadoop-yarn-common in the patch passed. +1 javadoc 0m 23s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235) +1 javadoc 0m 14s hadoop-yarn-site in the patch passed. +1 unit 2m 28s hadoop-yarn-common in the patch passed. +1 unit 14m 8s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 13s hadoop-yarn-site in the patch passed. +1 asflicense 0m 31s The patch does not generate ASF License warnings. 57m 12s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841120/YARN-5849.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 3e78ee482e1c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4fca94f Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14129/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14129/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Sorry, just caught one more thing. initializePreMountedCGroupController() needs javadocs.

        Show
        templedf Daniel Templeton added a comment - Sorry, just caught one more thing. initializePreMountedCGroupController() needs javadocs.
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Indeed. Thank you, Daniel Templeton. I added the javadocs

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Indeed. Thank you, Daniel Templeton . I added the javadocs
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 6m 53s trunk passed
        +1 compile 4m 57s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 27s trunk passed
        +1 mvneclipse 0m 54s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 1m 11s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 0m 57s the patch passed
        +1 compile 4m 36s the patch passed
        +1 javac 4m 36s the patch passed
        +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49)
        +1 mvnsite 1m 26s the patch passed
        +1 mvneclipse 0m 50s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 2m 2s the patch passed
        +1 javadoc 0m 33s hadoop-yarn-common in the patch passed.
        +1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235)
        +1 javadoc 0m 14s hadoop-yarn-site in the patch passed.
        +1 unit 2m 26s hadoop-yarn-common in the patch passed.
        +1 unit 13m 40s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 14s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        55m 16s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841782/YARN-5849.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
        uname Linux 6c1d4de0e871 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 291df5c
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14184/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14184/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 53s trunk passed +1 compile 4m 57s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 54s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 46s trunk passed +1 javadoc 1m 11s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 4m 36s the patch passed +1 javac 4m 36s the patch passed +1 checkstyle 0m 43s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 23 fixed = 26 total (was 49) +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 2s the patch passed +1 javadoc 0m 33s hadoop-yarn-common in the patch passed. +1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235) +1 javadoc 0m 14s hadoop-yarn-site in the patch passed. +1 unit 2m 26s hadoop-yarn-common in the patch passed. +1 unit 13m 40s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 14s hadoop-yarn-site in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 55m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841782/YARN-5849.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 6c1d4de0e871 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 291df5c Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14184/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14184/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        The last patch looks good to me. Bibin A Chundatt, want to do one last review before I +1 and commit?

        Show
        templedf Daniel Templeton added a comment - The last patch looks good to me. Bibin A Chundatt , want to do one last review before I +1 and commit?
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Daniel Templeton/Miklos Szegedi sorry for delay.

        Few comments

        1. Following check in CGroupsHandlerImpl#mountCGroupController is not required since check is already done
              if (!enableCGroupMount) {
                LOG.warn("CGroup mounting is disabled - ignoring mount request for: " +
                    controller.getName());
                return;
              }
          
        2. Comment could be modified
                // Make sure that Yarn cgroup path exists
                initializePreMountedCGroupController(controller)
          

          Can we change to ""Make sure that Yarn cgroup hierarchy path exists""

        Query

        Daniel Templeton/Miklos Szegedi

        yarnHierarchy.mkdir() would create directory based on the umask of user with which NM is started.
        With umask of 022 LCE container launch should be fine.
        But cluster steps with umask of NM start user as 077 drwx------ would be folder rights.This could cause permission issues rt??

        Not related to current patch

        1. TrafficControlBandwidthHandlerImpl can remove unused imports
        Show
        bibinchundatt Bibin A Chundatt added a comment - Daniel Templeton / Miklos Szegedi sorry for delay. Few comments Following check in CGroupsHandlerImpl#mountCGroupController is not required since check is already done if (!enableCGroupMount) { LOG.warn( "CGroup mounting is disabled - ignoring mount request for : " + controller.getName()); return ; } Comment could be modified // Make sure that Yarn cgroup path exists initializePreMountedCGroupController(controller) Can we change to ""Make sure that Yarn cgroup hierarchy path exists"" Query Daniel Templeton / Miklos Szegedi yarnHierarchy.mkdir() would create directory based on the umask of user with which NM is started. With umask of 022 LCE container launch should be fine. But cluster steps with umask of NM start user as 077 drwx------ would be folder rights.This could cause permission issues rt?? Not related to current patch TrafficControlBandwidthHandlerImpl can remove unused imports
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you, Bibin A Chundatt for the review! I addressed the three comments that you had in the next patch.
        In terms of umask, it should not be an issue in this case, since the node manager creates and uses the directories itself. The cgroup directory is not passed to the container.
        This raises an interesting question though. Probably we want to change File.mkdir() to limit the permissions to "rwx------" on all YARN cgroups directories regardless of umask. But since this affects the existing createCGroup method as well, I would address it in another Jira.

              Files.createDirectory(path, PosixFilePermissions
                  .asFileAttribute(PosixFilePermissions.fromString("rwx------")));
        
        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you, Bibin A Chundatt for the review! I addressed the three comments that you had in the next patch. In terms of umask, it should not be an issue in this case, since the node manager creates and uses the directories itself. The cgroup directory is not passed to the container. This raises an interesting question though. Probably we want to change File.mkdir() to limit the permissions to "rwx------" on all YARN cgroups directories regardless of umask. But since this affects the existing createCGroup method as well, I would address it in another Jira. Files.createDirectory(path, PosixFilePermissions .asFileAttribute(PosixFilePermissions.fromString( "rwx------" )));
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 7m 14s trunk passed
        +1 compile 4m 51s trunk passed
        +1 checkstyle 0m 43s trunk passed
        +1 mvnsite 1m 29s trunk passed
        +1 mvneclipse 0m 51s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 1m 10s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 1s the patch passed
        +1 compile 4m 40s the patch passed
        +1 javac 4m 40s the patch passed
        +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 25 unchanged - 24 fixed = 25 total (was 49)
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
        +1 findbugs 2m 21s the patch passed
        +1 javadoc 0m 34s hadoop-yarn-common in the patch passed.
        +1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235)
        +1 javadoc 0m 13s hadoop-yarn-site in the patch passed.
        +1 unit 2m 24s hadoop-yarn-common in the patch passed.
        +1 unit 13m 0s hadoop-yarn-server-nodemanager in the patch passed.
        +1 unit 0m 12s hadoop-yarn-site in the patch passed.
        +1 asflicense 0m 32s The patch does not generate ASF License warnings.
        55m 2s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5849
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842628/YARN-5849.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
        uname Linux 50d0e184d5fb 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b606e02
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14248/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14248/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 7m 14s trunk passed +1 compile 4m 51s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 1m 29s trunk passed +1 mvneclipse 0m 51s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 46s trunk passed +1 javadoc 1m 10s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 1s the patch passed +1 compile 4m 40s the patch passed +1 javac 4m 40s the patch passed +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 25 unchanged - 24 fixed = 25 total (was 49) +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 21s the patch passed +1 javadoc 0m 34s hadoop-yarn-common in the patch passed. +1 javadoc 0m 22s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 230 unchanged - 5 fixed = 230 total (was 235) +1 javadoc 0m 13s hadoop-yarn-site in the patch passed. +1 unit 2m 24s hadoop-yarn-common in the patch passed. +1 unit 13m 0s hadoop-yarn-server-nodemanager in the patch passed. +1 unit 0m 12s hadoop-yarn-site in the patch passed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 55m 2s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5849 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842628/YARN-5849.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 50d0e184d5fb 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b606e02 Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14248/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/14248/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Latest patch looks good to me. Bibin A Chundatt, any additional comments?

        Show
        templedf Daniel Templeton added a comment - Latest patch looks good to me. Bibin A Chundatt , any additional comments?
        Hide
        templedf Daniel Templeton added a comment -

        Bibin A Chundatt, any additional comments?

        Show
        templedf Daniel Templeton added a comment - Bibin A Chundatt , any additional comments?
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Daniel Templeton
        Sorry for the delay. Will try to review the same by today.
        We are planning to push this change only to trunk rt??

        Show
        bibinchundatt Bibin A Chundatt added a comment - Daniel Templeton Sorry for the delay. Will try to review the same by today. We are planning to push this change only to trunk rt??
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Latest patch looks good to me too.

        Show
        bibinchundatt Bibin A Chundatt added a comment - Latest patch looks good to me too.
        Hide
        templedf Daniel Templeton added a comment -

        Excellent. Thanks, Bibin A Chundatt! +1 Committing soon.

        Show
        templedf Daniel Templeton added a comment - Excellent. Thanks, Bibin A Chundatt ! +1 Committing soon.
        Hide
        templedf Daniel Templeton added a comment -

        Thanks, Miklos Szegedi, for the patch and, Bibin A Chundatt, for the reviews. Committed to trunk and branch-2.

        Show
        templedf Daniel Templeton added a comment - Thanks, Miklos Szegedi , for the patch and, Bibin A Chundatt , for the reviews. Committed to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11111 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11111/)
        YARN-5849. Automatically create YARN control group for pre-mounted (templedf: rev e6f13fe5d1df8918ffc680d18f9d5576f38893a6)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerCgroups.md
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsMemoryResourceHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TrafficControlBandwidthHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsCpuResourceHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestTrafficControlBandwidthHandlerImpl.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsMemoryResourceHandlerImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11111 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11111/ ) YARN-5849 . Automatically create YARN control group for pre-mounted (templedf: rev e6f13fe5d1df8918ffc680d18f9d5576f38893a6) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerCgroups.md (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsMemoryResourceHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsCpuResourceHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TrafficControlBandwidthHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsCpuResourceHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestTrafficControlBandwidthHandlerImpl.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsMemoryResourceHandlerImpl.java
        Hide
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment -

        Thank you Bibin A Chundatt for the review and Daniel Templeton for the review and commit.

        Show
        miklos.szegedi@cloudera.com Miklos Szegedi added a comment - Thank you Bibin A Chundatt for the review and Daniel Templeton for the review and commit.

          People

          • Assignee:
            miklos.szegedi@cloudera.com Miklos Szegedi
            Reporter:
            miklos.szegedi@cloudera.com Miklos Szegedi
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development