Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Attachments

      1. YARN-2619-1.patch
        14 kB
        Wei Yan
      2. YARN-2619.002.patch
        46 kB
        Varun Vasudev
      3. YARN-2619.003.patch
        49 kB
        Varun Vasudev
      4. YARN-2619.004.patch
        52 kB
        Varun Vasudev

      Activity

        ywskycn Wei Yan added a comment -

        Prelim patch for reviewing. The patch is based on YARN-2618.

        ywskycn Wei Yan added a comment - Prelim patch for reviewing. The patch is based on YARN-2618 .
        vvasudev Varun Vasudev added a comment -

        YARN-3443 introduces a resource handler chain which cleans up a lot of the cgroup handling. I've modified the patch Wei uploaded to make it work with the new chain. This version of the patch just does a fair sharing of resources. Once we add support for scheduling, we can modify the weighting.

        sidharta-s - can you please review?

        vvasudev Varun Vasudev added a comment - YARN-3443 introduces a resource handler chain which cleans up a lot of the cgroup handling. I've modified the patch Wei uploaded to make it work with the new chain. This version of the patch just does a fair sharing of resources. Once we add support for scheduling, we can modify the weighting. sidharta-s - can you please review?
        vvasudev Varun Vasudev added a comment -

        Uploaded the same patch with the name fixed.

        vvasudev Varun Vasudev added a comment - Uploaded the same patch with the name fixed.
        vvasudev Varun Vasudev added a comment -

        Forgot to mention,

        1. The new patch refactors the tests added by Wei for the disk scheduling as well as the mtab parsing.
        2. Some of the modifications to CgroupsLCEResourceHandler are no longer required since YARN-3443 moves that work to a common class.
        3. A new class has been added to handle the weight assigning for disk resources.
        vvasudev Varun Vasudev added a comment - Forgot to mention, The new patch refactors the tests added by Wei for the disk scheduling as well as the mtab parsing. Some of the modifications to CgroupsLCEResourceHandler are no longer required since YARN-3443 moves that work to a common class. A new class has been added to handle the weight assigning for disk resources.
        ywskycn Wei Yan added a comment -

        Thanks, vvasudev.

        ywskycn Wei Yan added a comment - Thanks, vvasudev .
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 32s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 30s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 5m 31s The applied patch generated 5 additional checkstyle issues.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api.
        +1 yarn tests 6m 6s Tests passed in hadoop-yarn-server-nodemanager.
            48m 34s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12728463/YARN-2619.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 7f07c4d
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7512/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7512/console

        This message was automatically generated.

        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 30s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 31s The applied patch generated 5 additional checkstyle issues. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api. +1 yarn tests 6m 6s Tests passed in hadoop-yarn-server-nodemanager.     48m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728463/YARN-2619.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7f07c4d checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7512/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7512/console This message was automatically generated.

        Thanks for the patch, vvasudev. Overall, the patch seems good - my only concern is about a dependency from TestCgroupsHandlerImpl to TestCgroupsLCEResourceHandler. More details inline:

        YarnConfiguration.java

        There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes.

        CGroupsHandler.java

        public static final String CGROUP_FILE_TASKS = "tasks”;
        public static final String CGROUP_PARAM_CLASSID = "classid";
        String CGROUP_PARAM_BLKIO = "weight”;
        

        Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT.

        CGroupsHandlerImpl.java

        private void init() throws ResourceHandlerException {
          initializeControllerPaths();
        }
        

        Extra empty line before this function.

        @VisibleForTesting
        Map<CGroupController, String> getControllerPaths() {
          return controllerPaths;
        }
        

        Private, modifiable state is being exposed here - use an unmodifiable map.

        CGroupsLCEResourceHandler.java

        @VisibleForTesting
        Map<CGroupController, String> getControllerPaths() {
          return controllerPaths;
        }
        

        Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map.

        Test CGroupsHandlerImpl.java

        File parentDir = new File(tmpPath);
        // create mock cgroup
        File cpuCgroupMountDir =
            TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu",
              hierarchy);
        Assert.assertTrue(cpuCgroupMountDir.exists());
        File blkioCgroupMountDir =
            TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir,
              "blkio", hierarchy);
        Assert.assertTrue(blkioCgroupMountDir.exists());
        File mockMtabFile =
            TestCgroupsLCEResourcesHandler.createMockMTab(parentDir);
        MyCgroupsHandler handler =
            new MyCgroupsHandler(conf, privilegedOperationExecutorMock);
        

        IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl.
        This file also has some unrelated formatting changes.

        @VisibleForTesting
        static void nullifyResourceHandlerChain() throws ResourceHandlerException {
        

        Can this be made private instead of package-private? This shouldn’t be available for use except for testing.

        CGroupsDiskResourceHandlerImpl.java

        Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here.

        String[] lines = data.split("\n”);
        

        Use %n or System.lineSeperator() ?

        if (partition.startsWith("sd") || partition.startsWith("hd")) {
        

        Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example.

        PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups="
          + cGroupsHandler.getPathForCGroupTasks(
        

        Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ?

        TestCGroupsDiskResourceHandlerImpl.java

        Assert.assertEquals("cgroups=" + path, args.get(0));
        

        Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.

        sidharta-s Sidharta Seethana added a comment - Thanks for the patch, vvasudev . Overall, the patch seems good - my only concern is about a dependency from TestCgroupsHandlerImpl to TestCgroupsLCEResourceHandler. More details inline: YarnConfiguration.java There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes. CGroupsHandler.java public static final String CGROUP_FILE_TASKS = "tasks”; public static final String CGROUP_PARAM_CLASSID = "classid" ; String CGROUP_PARAM_BLKIO = "weight”; Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT. CGroupsHandlerImpl.java private void init() throws ResourceHandlerException { initializeControllerPaths(); } Extra empty line before this function. @VisibleForTesting Map<CGroupController, String > getControllerPaths() { return controllerPaths; } Private, modifiable state is being exposed here - use an unmodifiable map. CGroupsLCEResourceHandler.java @VisibleForTesting Map<CGroupController, String > getControllerPaths() { return controllerPaths; } Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map. Test CGroupsHandlerImpl.java File parentDir = new File(tmpPath); // create mock cgroup File cpuCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu" , hierarchy); Assert.assertTrue(cpuCgroupMountDir.exists()); File blkioCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "blkio" , hierarchy); Assert.assertTrue(blkioCgroupMountDir.exists()); File mockMtabFile = TestCgroupsLCEResourcesHandler.createMockMTab(parentDir); MyCgroupsHandler handler = new MyCgroupsHandler(conf, privilegedOperationExecutorMock); IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542 ). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl. This file also has some unrelated formatting changes. @VisibleForTesting static void nullifyResourceHandlerChain() throws ResourceHandlerException { Can this be made private instead of package-private? This shouldn’t be available for use except for testing. CGroupsDiskResourceHandlerImpl.java Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here. String [] lines = data.split("\n”); Use %n or System.lineSeperator() ? if (partition.startsWith( "sd" ) || partition.startsWith( "hd" )) { Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example. PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups=" + cGroupsHandler.getPathForCGroupTasks( Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ? TestCGroupsDiskResourceHandlerImpl.java Assert.assertEquals( "cgroups=" + path, args.get(0)); Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.
        vvasudev Varun Vasudev added a comment -

        YarnConfiguration.java
        There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes.
        CGroupsHandler.java

        It fixes some checkstyle warnings about lines being more than 80 characters in length. Let me know if you feel strongly about it and I'll revert them.

        public static final String CGROUP_FILE_TASKS = "tasks”;
        public static final String CGROUP_PARAM_CLASSID = "classid";
        String CGROUP_PARAM_BLKIO = "weight”;

        Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT.

        Changed to public static final and renamed to CGROUP_PARAM_BLKIO_WEIGHT.

        CGroupsHandlerImpl.java

        private void init() throws ResourceHandlerException {
        initializeControllerPaths();
        }

        Extra empty line before this function.

        Fixed.

        @VisibleForTesting
        Map<CGroupController, String> getControllerPaths() {
        return controllerPaths;
        }

        Private, modifiable state is being exposed here - use an unmodifiable map.
        CGroupsLCEResourceHandler.java

        @VisibleForTesting
        Map<CGroupController, String> getControllerPaths() {
        return controllerPaths;
        }

        Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map.

        Fixed both.

        Test CGroupsHandlerImpl.java

        File parentDir = new File(tmpPath);
        // create mock cgroup
        File cpuCgroupMountDir =
        TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu",
        hierarchy);
        Assert.assertTrue(cpuCgroupMountDir.exists());
        File blkioCgroupMountDir =
        TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir,
        "blkio", hierarchy);
        Assert.assertTrue(blkioCgroupMountDir.exists());
        File mockMtabFile =
        TestCgroupsLCEResourcesHandler.createMockMTab(parentDir);
        MyCgroupsHandler handler =
        new MyCgroupsHandler(conf, privilegedOperationExecutorMock);

        IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl.
        This file also has some unrelated formatting changes.

        Good point. Moved the functions to TestCGroupsHandlerImpl. The formatting changes are similar to the ones in YarnConfiguration - fixing some checkstyle warnings.

        @VisibleForTesting
        static void nullifyResourceHandlerChain() throws ResourceHandlerException {

        Can this be made private instead of package-private? This shouldn’t be available for use except for testing.

        If it's made private, the tests can't reset the resource handler chain. The annotation isn't loaded into the JVM. If you have any other way to reset the chain, let me know.

        CGroupsDiskResourceHandlerImpl.java

        Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here.

        Changed it to CGroupsBlkioResourceHandler. Is that ok?

        String[] lines = data.split("\n”);

        Use %n or System.lineSeperator() ?

        Fixed.

        if (partition.startsWith("sd") || partition.startsWith("hd")) {

        Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example.

        Added vd and xvd prefix checks.

        PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups="
        + cGroupsHandler.getPathForCGroupTasks(

        Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ?
        TestCGroupsDiskResourceHandlerImpl.java

        Assert.assertEquals("cgroups=" + path, args.get(0));

        Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.

        Fixed.

        vvasudev Varun Vasudev added a comment - YarnConfiguration.java There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes. CGroupsHandler.java It fixes some checkstyle warnings about lines being more than 80 characters in length. Let me know if you feel strongly about it and I'll revert them. public static final String CGROUP_FILE_TASKS = "tasks”; public static final String CGROUP_PARAM_CLASSID = "classid"; String CGROUP_PARAM_BLKIO = "weight”; Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT. Changed to public static final and renamed to CGROUP_PARAM_BLKIO_WEIGHT. CGroupsHandlerImpl.java private void init() throws ResourceHandlerException { initializeControllerPaths(); } Extra empty line before this function. Fixed. @VisibleForTesting Map<CGroupController, String> getControllerPaths() { return controllerPaths; } Private, modifiable state is being exposed here - use an unmodifiable map. CGroupsLCEResourceHandler.java @VisibleForTesting Map<CGroupController, String> getControllerPaths() { return controllerPaths; } Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map. Fixed both. Test CGroupsHandlerImpl.java File parentDir = new File(tmpPath); // create mock cgroup File cpuCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu", hierarchy); Assert.assertTrue(cpuCgroupMountDir.exists()); File blkioCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "blkio", hierarchy); Assert.assertTrue(blkioCgroupMountDir.exists()); File mockMtabFile = TestCgroupsLCEResourcesHandler.createMockMTab(parentDir); MyCgroupsHandler handler = new MyCgroupsHandler(conf, privilegedOperationExecutorMock); IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542 ). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl. This file also has some unrelated formatting changes. Good point. Moved the functions to TestCGroupsHandlerImpl. The formatting changes are similar to the ones in YarnConfiguration - fixing some checkstyle warnings. @VisibleForTesting static void nullifyResourceHandlerChain() throws ResourceHandlerException { Can this be made private instead of package-private? This shouldn’t be available for use except for testing. If it's made private, the tests can't reset the resource handler chain. The annotation isn't loaded into the JVM. If you have any other way to reset the chain, let me know. CGroupsDiskResourceHandlerImpl.java Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here. Changed it to CGroupsBlkioResourceHandler. Is that ok? String[] lines = data.split("\n”); Use %n or System.lineSeperator() ? Fixed. if (partition.startsWith("sd") || partition.startsWith("hd")) { Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example. Added vd and xvd prefix checks. PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups=" + cGroupsHandler.getPathForCGroupTasks( Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ? TestCGroupsDiskResourceHandlerImpl.java Assert.assertEquals("cgroups=" + path, args.get(0)); Use PrivilegedOperation.CGROUP_ARG_PREFIX instead. Fixed.
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 56s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 7m 43s There were no new javac warning messages.
        +1 javadoc 9m 39s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 5m 27s The applied patch generated 5 additional checkstyle issues.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
        +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager.
            49m 5s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12728769/YARN-2619.003.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 99fe03e
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7522/testReport/
        Java 1.7.0_55
        uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7522/console

        This message was automatically generated.

        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 27s The applied patch generated 5 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager.     49m 5s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728769/YARN-2619.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 99fe03e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7522/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7522/console This message was automatically generated.

        vvasudev , the changes you made seem mine. There is one minor thing in the new patch - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl . This can probably be removed at the time the commit goes in.
        +1 to this patch.

        vinodkv , can you please take a look at this patch?

        sidharta-s Sidharta Seethana added a comment - vvasudev , the changes you made seem mine. There is one minor thing in the new patch - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl . This can probably be removed at the time the commit goes in. +1 to this patch. vinodkv , can you please take a look at this patch?

        A very clean patch!

        This is essentially a rewrite. FYI ywskycn.

        Seems like we are also not yet tying this for scheduling, that's my expectation too. But add a note on this JIRA as to how sharing happens? And the defaults (what does a weight of 500 mean?).

        Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there?

        CGroupsDiskResourceHandlerImpl

        • Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface
        • Print a warning if PARTITIONS_FILE cannot be read?
        vinodkv Vinod Kumar Vavilapalli added a comment - A very clean patch! This is essentially a rewrite. FYI ywskycn . Seems like we are also not yet tying this for scheduling, that's my expectation too. But add a note on this JIRA as to how sharing happens? And the defaults (what does a weight of 500 mean?). Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there? CGroupsDiskResourceHandlerImpl Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface Print a warning if PARTITIONS_FILE cannot be read?
        vvasudev Varun Vasudev added a comment -

        The sharing is just an equal split of disk operations. The number 500 is just arbitrary - all that matters is that all containers get assigned the same weight which ensures they get an equal share of disk operations. Once we have scheduling support, the weight will be determined dynamically based on the allocated resources.

        vvasudev Varun Vasudev added a comment - The sharing is just an equal split of disk operations. The number 500 is just arbitrary - all that matters is that all containers get assigned the same weight which ensures they get an equal share of disk operations. Once we have scheduling support, the weight will be determined dynamically based on the allocated resources.
        vvasudev Varun Vasudev added a comment -

        there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl

        Fixed.

        And the defaults (what does a weight of 500 mean?)

        Added a comment in the implementation that it's just arbitrary.

        Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there?

        We should do it once YARN-3542 gets committed.

        Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface

        Fixed.

        Print a warning if PARTITIONS_FILE cannot be read?

        Fixed.

        I've also re-factored part of CGroupsHandlerImpl to make testing cleaner(removing the need to read the controllerPaths map).

        sidharta-s - can you please review to make sure the refactoring is ok?

        vvasudev Varun Vasudev added a comment - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl Fixed. And the defaults (what does a weight of 500 mean?) Added a comment in the implementation that it's just arbitrary. Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there? We should do it once YARN-3542 gets committed. Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface Fixed. Print a warning if PARTITIONS_FILE cannot be read? Fixed. I've also re-factored part of CGroupsHandlerImpl to make testing cleaner(removing the need to read the controllerPaths map). sidharta-s - can you please review to make sure the refactoring is ok?
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 46s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 javac 8m 57s There were no new javac warning messages.
        +1 javadoc 11m 4s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 4m 57s The applied patch generated 5 additional checkstyle issues.
        +1 install 1m 43s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 2m 23s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api.
        +1 yarn tests 6m 11s Tests passed in hadoop-yarn-server-nodemanager.
            55m 21s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12729166/YARN-2619.004.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8f82970
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/checkstyle-result-diff.txt
        hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-api.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7541/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/7541/console

        This message was automatically generated.

        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 46s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 8m 57s There were no new javac warning messages. +1 javadoc 11m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 4m 57s The applied patch generated 5 additional checkstyle issues. +1 install 1m 43s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 23s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api. +1 yarn tests 6m 11s Tests passed in hadoop-yarn-server-nodemanager.     55m 21s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12729166/YARN-2619.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f82970 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7541/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7541/console This message was automatically generated.

        vvasudev , thanks for refactoring the test to be cleaner. The corresponding changes seem good to me.

        sidharta-s Sidharta Seethana added a comment - vvasudev , thanks for refactoring the test to be cleaner. The corresponding changes seem good to me.

        Looks good to me too, +1. Checking this in..

        vinodkv Vinod Kumar Vavilapalli added a comment - Looks good to me too, +1. Checking this in..

        Committed this to trunk and branch-2. Thanks Varun!

        Tx to Wei for the initial patch and the test-cases. And Sid for the reviews!

        vinodkv Vinod Kumar Vavilapalli added a comment - Committed this to trunk and branch-2. Thanks Varun! Tx to Wei for the initial patch and the test-cases. And Sid for the reviews!
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #7711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7711/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7711/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/180/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/180/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #914 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/914/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #914 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/914/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2112 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2112/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2112 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2112/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/171/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/171/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/181/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/181/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2130 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2130/)
        YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/CHANGES.txt
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2130 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2130/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt

        I really would have liked to review or at least skim through the patch before it went in. I should have been watching this issue, but would have appreciated a ping especially given the patrons were aware of my involvement on YARN-2139 and YARN-2168.

        Also, given this is a sub-task of YARN-2139 and we have decided on doing it on a branch, I am surprised we committed this to trunk and branch-2. Is it too late to move it to the branch so we can look at all disk IO together?

        kasha Karthik Kambatla added a comment - I really would have liked to review or at least skim through the patch before it went in. I should have been watching this issue, but would have appreciated a ping especially given the patrons were aware of my involvement on YARN-2139 and YARN-2168 . Also, given this is a sub-task of YARN-2139 and we have decided on doing it on a branch, I am surprised we committed this to trunk and branch-2. Is it too late to move it to the branch so we can look at all disk IO together?

        I pinged Wei above and waited for a few days. I should have pinged you but missed, my bad.

        It wasn't originally clear to me that this was going to a branch. Hence the confusion I expressed over at YARN-2618.

        What's done is done. Clearly the patch here made sure everything is private, so I wouldn't be worried about removing it from trunk/branch-2. So I propose we create a new branch for YARN-2139 and proceed work there from this point of time.

        vinodkv Vinod Kumar Vavilapalli added a comment - I pinged Wei above and waited for a few days. I should have pinged you but missed, my bad. It wasn't originally clear to me that this was going to a branch. Hence the confusion I expressed over at YARN-2618 . What's done is done. Clearly the patch here made sure everything is private, so I wouldn't be worried about removing it from trunk/branch-2. So I propose we create a new branch for YARN-2139 and proceed work there from this point of time.

        People

          vvasudev Varun Vasudev
          ywskycn Wei Yan
          Votes:
          0 Vote for this issue
          Watchers:
          19 Start watching this issue

          Dates

            Created:
            Updated:
            Resolved: