Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. YARN-2619.002.patch
      46 kB
      Varun Vasudev
    2. YARN-2619.003.patch
      49 kB
      Varun Vasudev
    3. YARN-2619.004.patch
      52 kB
      Varun Vasudev
    4. YARN-2619-1.patch
      14 kB
      Wei Yan

      Activity

      Hide
      ywskycn Wei Yan added a comment -

      Prelim patch for reviewing. The patch is based on YARN-2618.

      Show
      ywskycn Wei Yan added a comment - Prelim patch for reviewing. The patch is based on YARN-2618 .
      Hide
      vvasudev Varun Vasudev added a comment -

      YARN-3443 introduces a resource handler chain which cleans up a lot of the cgroup handling. I've modified the patch Wei uploaded to make it work with the new chain. This version of the patch just does a fair sharing of resources. Once we add support for scheduling, we can modify the weighting.

      Sidharta Seethana - can you please review?

      Show
      vvasudev Varun Vasudev added a comment - YARN-3443 introduces a resource handler chain which cleans up a lot of the cgroup handling. I've modified the patch Wei uploaded to make it work with the new chain. This version of the patch just does a fair sharing of resources. Once we add support for scheduling, we can modify the weighting. Sidharta Seethana - can you please review?
      Hide
      vvasudev Varun Vasudev added a comment -

      Uploaded the same patch with the name fixed.

      Show
      vvasudev Varun Vasudev added a comment - Uploaded the same patch with the name fixed.
      Hide
      vvasudev Varun Vasudev added a comment -

      Forgot to mention,

      1. The new patch refactors the tests added by Wei for the disk scheduling as well as the mtab parsing.
      2. Some of the modifications to CgroupsLCEResourceHandler are no longer required since YARN-3443 moves that work to a common class.
      3. A new class has been added to handle the weight assigning for disk resources.
      Show
      vvasudev Varun Vasudev added a comment - Forgot to mention, The new patch refactors the tests added by Wei for the disk scheduling as well as the mtab parsing. Some of the modifications to CgroupsLCEResourceHandler are no longer required since YARN-3443 moves that work to a common class. A new class has been added to handle the weight assigning for disk resources.
      Hide
      ywskycn Wei Yan added a comment -

      Thanks, Varun Vasudev.

      Show
      ywskycn Wei Yan added a comment - Thanks, Varun Vasudev .
      Hide
      hadoopqa Hadoop QA added a comment -



      -1 overall



      Vote Subsystem Runtime Comment
      0 pre-patch 14m 32s Pre-patch trunk compilation is healthy.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
      +1 whitespace 0m 0s The patch has no lines that end in whitespace.
      +1 javac 7m 32s There were no new javac warning messages.
      +1 javadoc 9m 30s There were no new javadoc warning messages.
      +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
      -1 checkstyle 5m 31s The applied patch generated 5 additional checkstyle issues.
      +1 install 1m 31s mvn install still works.
      +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
      +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
      +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api.
      +1 yarn tests 6m 6s Tests passed in hadoop-yarn-server-nodemanager.
          48m 34s  



      Subsystem Report/Notes
      Patch URL http://issues.apache.org/jira/secure/attachment/12728463/YARN-2619.002.patch
      Optional Tests javadoc javac unit findbugs checkstyle
      git revision trunk / 7f07c4d
      checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/checkstyle-result-diff.txt
      hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-api.txt
      hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
      Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7512/testReport/
      Java 1.7.0_55
      uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
      Console output https://builds.apache.org/job/PreCommit-YARN-Build/7512/console

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 30s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 31s The applied patch generated 5 additional checkstyle issues. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api. +1 yarn tests 6m 6s Tests passed in hadoop-yarn-server-nodemanager.     48m 34s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728463/YARN-2619.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7f07c4d checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7512/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7512/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7512/console This message was automatically generated.
      Hide
      sidharta-s Sidharta Seethana added a comment -

      Thanks for the patch, Varun Vasudev. Overall, the patch seems good - my only concern is about a dependency from TestCgroupsHandlerImpl to TestCgroupsLCEResourceHandler. More details inline:

      YarnConfiguration.java

      There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes.

      CGroupsHandler.java

      public static final String CGROUP_FILE_TASKS = "tasks”;
      public static final String CGROUP_PARAM_CLASSID = "classid";
      String CGROUP_PARAM_BLKIO = "weight”;
      

      Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT.

      CGroupsHandlerImpl.java

      private void init() throws ResourceHandlerException {
        initializeControllerPaths();
      }
      

      Extra empty line before this function.

      @VisibleForTesting
      Map<CGroupController, String> getControllerPaths() {
        return controllerPaths;
      }
      

      Private, modifiable state is being exposed here - use an unmodifiable map.

      CGroupsLCEResourceHandler.java

      @VisibleForTesting
      Map<CGroupController, String> getControllerPaths() {
        return controllerPaths;
      }
      

      Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map.

      Test CGroupsHandlerImpl.java

      File parentDir = new File(tmpPath);
      // create mock cgroup
      File cpuCgroupMountDir =
          TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu",
            hierarchy);
      Assert.assertTrue(cpuCgroupMountDir.exists());
      File blkioCgroupMountDir =
          TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir,
            "blkio", hierarchy);
      Assert.assertTrue(blkioCgroupMountDir.exists());
      File mockMtabFile =
          TestCgroupsLCEResourcesHandler.createMockMTab(parentDir);
      MyCgroupsHandler handler =
          new MyCgroupsHandler(conf, privilegedOperationExecutorMock);
      

      IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl.
      This file also has some unrelated formatting changes.

      @VisibleForTesting
      static void nullifyResourceHandlerChain() throws ResourceHandlerException {
      

      Can this be made private instead of package-private? This shouldn’t be available for use except for testing.

      CGroupsDiskResourceHandlerImpl.java

      Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here.

      String[] lines = data.split("\n”);
      

      Use %n or System.lineSeperator() ?

      if (partition.startsWith("sd") || partition.startsWith("hd")) {
      

      Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example.

      PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups="
        + cGroupsHandler.getPathForCGroupTasks(
      

      Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ?

      TestCGroupsDiskResourceHandlerImpl.java

      Assert.assertEquals("cgroups=" + path, args.get(0));
      

      Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.

      Show
      sidharta-s Sidharta Seethana added a comment - Thanks for the patch, Varun Vasudev . Overall, the patch seems good - my only concern is about a dependency from TestCgroupsHandlerImpl to TestCgroupsLCEResourceHandler. More details inline: YarnConfiguration.java There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes. CGroupsHandler.java public static final String CGROUP_FILE_TASKS = "tasks”; public static final String CGROUP_PARAM_CLASSID = "classid" ; String CGROUP_PARAM_BLKIO = "weight”; Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT. CGroupsHandlerImpl.java private void init() throws ResourceHandlerException { initializeControllerPaths(); } Extra empty line before this function. @VisibleForTesting Map<CGroupController, String > getControllerPaths() { return controllerPaths; } Private, modifiable state is being exposed here - use an unmodifiable map. CGroupsLCEResourceHandler.java @VisibleForTesting Map<CGroupController, String > getControllerPaths() { return controllerPaths; } Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map. Test CGroupsHandlerImpl.java File parentDir = new File(tmpPath); // create mock cgroup File cpuCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu" , hierarchy); Assert.assertTrue(cpuCgroupMountDir.exists()); File blkioCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "blkio" , hierarchy); Assert.assertTrue(blkioCgroupMountDir.exists()); File mockMtabFile = TestCgroupsLCEResourcesHandler.createMockMTab(parentDir); MyCgroupsHandler handler = new MyCgroupsHandler(conf, privilegedOperationExecutorMock); IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542 ). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl. This file also has some unrelated formatting changes. @VisibleForTesting static void nullifyResourceHandlerChain() throws ResourceHandlerException { Can this be made private instead of package-private? This shouldn’t be available for use except for testing. CGroupsDiskResourceHandlerImpl.java Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here. String [] lines = data.split("\n”); Use %n or System.lineSeperator() ? if (partition.startsWith( "sd" ) || partition.startsWith( "hd" )) { Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example. PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups=" + cGroupsHandler.getPathForCGroupTasks( Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ? TestCGroupsDiskResourceHandlerImpl.java Assert.assertEquals( "cgroups=" + path, args.get(0)); Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.
      Hide
      vvasudev Varun Vasudev added a comment -

      YarnConfiguration.java
      There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes.
      CGroupsHandler.java

      It fixes some checkstyle warnings about lines being more than 80 characters in length. Let me know if you feel strongly about it and I'll revert them.

      public static final String CGROUP_FILE_TASKS = "tasks”;
      public static final String CGROUP_PARAM_CLASSID = "classid";
      String CGROUP_PARAM_BLKIO = "weight”;

      Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT.

      Changed to public static final and renamed to CGROUP_PARAM_BLKIO_WEIGHT.

      CGroupsHandlerImpl.java

      private void init() throws ResourceHandlerException

      Unknown macro: { initializeControllerPaths(); }

      Extra empty line before this function.

      Fixed.

      @VisibleForTesting
      Map<CGroupController, String> getControllerPaths()

      Unknown macro: { return controllerPaths; }

      Private, modifiable state is being exposed here - use an unmodifiable map.
      CGroupsLCEResourceHandler.java

      @VisibleForTesting
      Map<CGroupController, String> getControllerPaths()

      Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map.

      Fixed both.

      Test CGroupsHandlerImpl.java

      File parentDir = new File(tmpPath);
      // create mock cgroup
      File cpuCgroupMountDir =
      TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu",
      hierarchy);
      Assert.assertTrue(cpuCgroupMountDir.exists());
      File blkioCgroupMountDir =
      TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir,
      "blkio", hierarchy);
      Assert.assertTrue(blkioCgroupMountDir.exists());
      File mockMtabFile =
      TestCgroupsLCEResourcesHandler.createMockMTab(parentDir);
      MyCgroupsHandler handler =
      new MyCgroupsHandler(conf, privilegedOperationExecutorMock);

      IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl.
      This file also has some unrelated formatting changes.

      Good point. Moved the functions to TestCGroupsHandlerImpl. The formatting changes are similar to the ones in YarnConfiguration - fixing some checkstyle warnings.

      @VisibleForTesting
      static void nullifyResourceHandlerChain() throws ResourceHandlerException {

      Can this be made private instead of package-private? This shouldn’t be available for use except for testing.

      If it's made private, the tests can't reset the resource handler chain. The annotation isn't loaded into the JVM. If you have any other way to reset the chain, let me know.

      CGroupsDiskResourceHandlerImpl.java

      Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here.

      Changed it to CGroupsBlkioResourceHandler. Is that ok?

      String[] lines = data.split("\n”);

      Use %n or System.lineSeperator() ?

      Fixed.

      if (partition.startsWith("sd") || partition.startsWith("hd")) {

      Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example.

      Added vd and xvd prefix checks.

      PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups="
      + cGroupsHandler.getPathForCGroupTasks(

      Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ?
      TestCGroupsDiskResourceHandlerImpl.java

      Assert.assertEquals("cgroups=" + path, args.get(0));

      Use PrivilegedOperation.CGROUP_ARG_PREFIX instead.

      Fixed.

      Show
      vvasudev Varun Vasudev added a comment - YarnConfiguration.java There seem to be a bunch of unrelated formatting changes in YarnConfiguration.java - consider removing these changes. CGroupsHandler.java It fixes some checkstyle warnings about lines being more than 80 characters in length. Let me know if you feel strongly about it and I'll revert them. public static final String CGROUP_FILE_TASKS = "tasks”; public static final String CGROUP_PARAM_CLASSID = "classid"; String CGROUP_PARAM_BLKIO = "weight”; Make this string “public static final” to be consistent with the rest of the the strings here? It doesn’t seem like ‘weight’ is the only supported cgroup param for blkio. , consider re-naming accordingly ( e.g see classid above ). Alternatively, maybe rename both? CGROUP_PARAM_NET_CLS_CLASSID and CGROUP_PARAM_BLKIO_WEIGHT. Changed to public static final and renamed to CGROUP_PARAM_BLKIO_WEIGHT. CGroupsHandlerImpl.java private void init() throws ResourceHandlerException Unknown macro: { initializeControllerPaths(); } Extra empty line before this function. Fixed. @VisibleForTesting Map<CGroupController, String> getControllerPaths() Unknown macro: { return controllerPaths; } Private, modifiable state is being exposed here - use an unmodifiable map. CGroupsLCEResourceHandler.java @VisibleForTesting Map<CGroupController, String> getControllerPaths() Same comment as above. Private, modifiable state is being exposed here - use an unmodifiable map. Fixed both. Test CGroupsHandlerImpl.java File parentDir = new File(tmpPath); // create mock cgroup File cpuCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "cpu", hierarchy); Assert.assertTrue(cpuCgroupMountDir.exists()); File blkioCgroupMountDir = TestCgroupsLCEResourcesHandler.createMockCgroupMount(parentDir, "blkio", hierarchy); Assert.assertTrue(blkioCgroupMountDir.exists()); File mockMtabFile = TestCgroupsLCEResourcesHandler.createMockMTab(parentDir); MyCgroupsHandler handler = new MyCgroupsHandler(conf, privilegedOperationExecutorMock); IMO, it doesn’t make sense to have a dependency on TestCgroupsLCEResourceHandler here. Eventually, we want to get rid of CgroupsLCEResourceHandler (which is very CPU specific) and implement this using the new resource handler framework. CGroupsHandlerImpl is meant to be a replacement for (the cgroups functionality of) CgroupsLCEResourceHandler (See YARN-3542 ). Given that this is the case, we should consider implementing this functionality in TestCGroupsHandlerImpl. This file also has some unrelated formatting changes. Good point. Moved the functions to TestCGroupsHandlerImpl. The formatting changes are similar to the ones in YarnConfiguration - fixing some checkstyle warnings. @VisibleForTesting static void nullifyResourceHandlerChain() throws ResourceHandlerException { Can this be made private instead of package-private? This shouldn’t be available for use except for testing. If it's made private, the tests can't reset the resource handler chain. The annotation isn't loaded into the JVM. If you have any other way to reset the chain, let me know. CGroupsDiskResourceHandlerImpl.java Class Name : Maybe we should use something less ‘generic’ - (with CFQ?) I expect that we’ll have more disk/cgroup resource handlers in place in the future - so a less generic name might be better here. Changed it to CGroupsBlkioResourceHandler. Is that ok? String[] lines = data.split("\n”); Use %n or System.lineSeperator() ? Fixed. if (partition.startsWith("sd") || partition.startsWith("hd")) { Is there a reason to only use these prefixes? I have seen vd* on some virtualized envs, for example. Added vd and xvd prefix checks. PrivilegedOperation.OperationType.ADD_PID_TO_CGROUP, "cgroups=" + cGroupsHandler.getPathForCGroupTasks( Use PrivilegedOperation.CGROUP_ARG_PREFIX instead ? TestCGroupsDiskResourceHandlerImpl.java Assert.assertEquals("cgroups=" + path, args.get(0)); Use PrivilegedOperation.CGROUP_ARG_PREFIX instead. Fixed.
      Hide
      hadoopqa Hadoop QA added a comment -



      -1 overall



      Vote Subsystem Runtime Comment
      0 pre-patch 14m 56s Pre-patch trunk compilation is healthy.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
      +1 whitespace 0m 0s The patch has no lines that end in whitespace.
      +1 javac 7m 43s There were no new javac warning messages.
      +1 javadoc 9m 39s There were no new javadoc warning messages.
      +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
      -1 checkstyle 5m 27s The applied patch generated 5 additional checkstyle issues.
      +1 install 1m 33s mvn install still works.
      +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
      +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
      +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api.
      +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager.
          49m 5s  



      Subsystem Report/Notes
      Patch URL http://issues.apache.org/jira/secure/attachment/12728769/YARN-2619.003.patch
      Optional Tests javadoc javac unit findbugs checkstyle
      git revision trunk / 99fe03e
      checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/checkstyle-result-diff.txt
      hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-api.txt
      hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
      Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7522/testReport/
      Java 1.7.0_55
      uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
      Console output https://builds.apache.org/job/PreCommit-YARN-Build/7522/console

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 56s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 27s The applied patch generated 5 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 24s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 25s Tests passed in hadoop-yarn-api. +1 yarn tests 5m 50s Tests passed in hadoop-yarn-server-nodemanager.     49m 5s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728769/YARN-2619.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 99fe03e checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7522/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7522/testReport/ Java 1.7.0_55 uname Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7522/console This message was automatically generated.
      Hide
      sidharta-s Sidharta Seethana added a comment -

      Varun Vasudev , the changes you made seem mine. There is one minor thing in the new patch - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl . This can probably be removed at the time the commit goes in.
      +1 to this patch.

      Vinod Kumar Vavilapalli , can you please take a look at this patch?

      Show
      sidharta-s Sidharta Seethana added a comment - Varun Vasudev , the changes you made seem mine. There is one minor thing in the new patch - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl . This can probably be removed at the time the commit goes in. +1 to this patch. Vinod Kumar Vavilapalli , can you please take a look at this patch?
      Hide
      vinodkv Vinod Kumar Vavilapalli added a comment -

      A very clean patch!

      This is essentially a rewrite. FYI Wei Yan.

      Seems like we are also not yet tying this for scheduling, that's my expectation too. But add a note on this JIRA as to how sharing happens? And the defaults (what does a weight of 500 mean?).

      Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there?

      CGroupsDiskResourceHandlerImpl

      • Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface
      • Print a warning if PARTITIONS_FILE cannot be read?
      Show
      vinodkv Vinod Kumar Vavilapalli added a comment - A very clean patch! This is essentially a rewrite. FYI Wei Yan . Seems like we are also not yet tying this for scheduling, that's my expectation too. But add a note on this JIRA as to how sharing happens? And the defaults (what does a weight of 500 mean?). Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there? CGroupsDiskResourceHandlerImpl Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface Print a warning if PARTITIONS_FILE cannot be read?
      Hide
      vvasudev Varun Vasudev added a comment -

      The sharing is just an equal split of disk operations. The number 500 is just arbitrary - all that matters is that all containers get assigned the same weight which ensures they get an equal share of disk operations. Once we have scheduling support, the weight will be determined dynamically based on the allocated resources.

      Show
      vvasudev Varun Vasudev added a comment - The sharing is just an equal split of disk operations. The number 500 is just arbitrary - all that matters is that all containers get assigned the same weight which ensures they get an equal share of disk operations. Once we have scheduling support, the weight will be determined dynamically based on the allocated resources.
      Hide
      vvasudev Varun Vasudev added a comment -

      there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl

      Fixed.

      And the defaults (what does a weight of 500 mean?)

      Added a comment in the implementation that it's just arbitrary.

      Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there?

      We should do it once YARN-3542 gets committed.

      Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface

      Fixed.

      Print a warning if PARTITIONS_FILE cannot be read?

      Fixed.

      I've also re-factored part of CGroupsHandlerImpl to make testing cleaner(removing the need to read the controllerPaths map).

      Sidharta Seethana - can you please review to make sure the refactoring is ok?

      Show
      vvasudev Varun Vasudev added a comment - there is an unused import org.apache.hadoop.yarn.server.nodemanager.util.TestCgroupsLCEResourcesHandler in TestCGroupsHandlerImpl Fixed. And the defaults (what does a weight of 500 mean?) Added a comment in the implementation that it's just arbitrary. Should we deprecate LCEResourcesHandler hierarchy so that future work doesn't go there? We should do it once YARN-3542 gets committed. Add Override annotations for methods that are overridden so it is clear what behavior is dictated by the base interface Fixed. Print a warning if PARTITIONS_FILE cannot be read? Fixed. I've also re-factored part of CGroupsHandlerImpl to make testing cleaner(removing the need to read the controllerPaths map). Sidharta Seethana - can you please review to make sure the refactoring is ok?
      Hide
      hadoopqa Hadoop QA added a comment -



      -1 overall



      Vote Subsystem Runtime Comment
      0 pre-patch 18m 46s Pre-patch trunk compilation is healthy.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
      +1 whitespace 0m 0s The patch has no lines that end in whitespace.
      +1 javac 8m 57s There were no new javac warning messages.
      +1 javadoc 11m 4s There were no new javadoc warning messages.
      +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
      -1 checkstyle 4m 57s The applied patch generated 5 additional checkstyle issues.
      +1 install 1m 43s mvn install still works.
      +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
      +1 findbugs 2m 23s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
      +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api.
      +1 yarn tests 6m 11s Tests passed in hadoop-yarn-server-nodemanager.
          55m 21s  



      Subsystem Report/Notes
      Patch URL http://issues.apache.org/jira/secure/attachment/12729166/YARN-2619.004.patch
      Optional Tests javadoc javac unit findbugs checkstyle
      git revision trunk / 8f82970
      checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/checkstyle-result-diff.txt
      hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-api.txt
      hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
      Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7541/testReport/
      Java 1.7.0_55
      uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
      Console output https://builds.apache.org/job/PreCommit-YARN-Build/7541/console

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 46s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 8m 57s There were no new javac warning messages. +1 javadoc 11m 4s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 4m 57s The applied patch generated 5 additional checkstyle issues. +1 install 1m 43s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 23s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 yarn tests 0m 22s Tests passed in hadoop-yarn-api. +1 yarn tests 6m 11s Tests passed in hadoop-yarn-server-nodemanager.     55m 21s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12729166/YARN-2619.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8f82970 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/checkstyle-result-diff.txt hadoop-yarn-api test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-api.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/7541/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/7541/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/7541/console This message was automatically generated.
      Hide
      sidharta-s Sidharta Seethana added a comment -

      Varun Vasudev , thanks for refactoring the test to be cleaner. The corresponding changes seem good to me.

      Show
      sidharta-s Sidharta Seethana added a comment - Varun Vasudev , thanks for refactoring the test to be cleaner. The corresponding changes seem good to me.
      Hide
      vinodkv Vinod Kumar Vavilapalli added a comment -

      Looks good to me too, +1. Checking this in..

      Show
      vinodkv Vinod Kumar Vavilapalli added a comment - Looks good to me too, +1. Checking this in..
      Hide
      vinodkv Vinod Kumar Vavilapalli added a comment -

      Committed this to trunk and branch-2. Thanks Varun!

      Tx to Wei for the initial patch and the test-cases. And Sid for the reviews!

      Show
      vinodkv Vinod Kumar Vavilapalli added a comment - Committed this to trunk and branch-2. Thanks Varun! Tx to Wei for the initial patch and the test-cases. And Sid for the reviews!
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Hadoop-trunk-Commit #7711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7711/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7711 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7711/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/180/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #180 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/180/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Yarn-trunk #914 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/914/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #914 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/914/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Hdfs-trunk #2112 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2112/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2112 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2112/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/171/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #171 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/171/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      Hide
      hudson Hudson added a comment -

      FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/181/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/CHANGES.txt
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      Show
      hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #181 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/181/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2130 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2130/)
      YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)

      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java
      • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
      • hadoop-yarn-project/CHANGES.txt
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2130 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2130/ ) YARN-2619 . Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. (vinodkv: rev 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/util/CgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/DiskResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestResourceHandlerModule.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestCgroupsLCEResourcesHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsBlkioResourceHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/CHANGES.txt
      Hide
      kasha Karthik Kambatla added a comment -

      I really would have liked to review or at least skim through the patch before it went in. I should have been watching this issue, but would have appreciated a ping especially given the patrons were aware of my involvement on YARN-2139 and YARN-2168.

      Also, given this is a sub-task of YARN-2139 and we have decided on doing it on a branch, I am surprised we committed this to trunk and branch-2. Is it too late to move it to the branch so we can look at all disk IO together?

      Show
      kasha Karthik Kambatla added a comment - I really would have liked to review or at least skim through the patch before it went in. I should have been watching this issue, but would have appreciated a ping especially given the patrons were aware of my involvement on YARN-2139 and YARN-2168 . Also, given this is a sub-task of YARN-2139 and we have decided on doing it on a branch, I am surprised we committed this to trunk and branch-2. Is it too late to move it to the branch so we can look at all disk IO together?
      Hide
      vinodkv Vinod Kumar Vavilapalli added a comment -

      I pinged Wei above and waited for a few days. I should have pinged you but missed, my bad.

      It wasn't originally clear to me that this was going to a branch. Hence the confusion I expressed over at YARN-2618.

      What's done is done. Clearly the patch here made sure everything is private, so I wouldn't be worried about removing it from trunk/branch-2. So I propose we create a new branch for YARN-2139 and proceed work there from this point of time.

      Show
      vinodkv Vinod Kumar Vavilapalli added a comment - I pinged Wei above and waited for a few days. I should have pinged you but missed, my bad. It wasn't originally clear to me that this was going to a branch. Hence the confusion I expressed over at YARN-2618 . What's done is done. Clearly the patch here made sure everything is private, so I wouldn't be worried about removing it from trunk/branch-2. So I propose we create a new branch for YARN-2139 and proceed work there from this point of time.

        People

        • Assignee:
          vvasudev Varun Vasudev
          Reporter:
          ywskycn Wei Yan
        • Votes:
          0 Vote for this issue
          Watchers:
          19 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development