Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2140 Add support for network IO isolation/scheduling for containers
  3. YARN-3443

Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      The current cgroups implementation is closely tied to supporting CPU as a resource . This patch separates out CGroups implementation into a reusable class as well as provides a simple ResourceHandler subsystem that will enable us to add support for new resource types on the NM - e.g Network, Disk etc.
      Show
      The current cgroups implementation is closely tied to supporting CPU as a resource . This patch separates out CGroups implementation into a reusable class as well as provides a simple ResourceHandler subsystem that will enable us to add support for new resource types on the NM - e.g Network, Disk etc.

      Description

      Today, support for CPU and memory as resources (on linux) are implemented in a way that cannot be easily extended to other new resource types (e.g network/disk). For example, some functionality cgroups functionality is implemented in LCE (mountCgroups) and the rest in CgroupsLCEResourcesHandler. CPU specific functionality is also implemented in CgroupsLCEResourcesHandler - using this handler automatically enables CPU as a resource. Some cgroups functionality requires elevated/super-user privileges and needs to be implemented via the container-executor binary. Implementing support for a new resource type in linux using the existing classes/mechanisms would be messy (for example, we might have to significantly modify/bloat CgroupsLCEResourceHandler). As an alternative, we have implemented a new ‘ResourceHandler’ mechanism that makes things cleaner and enables easier addition of new resource types. When adding support for a new resource type in the NM (from an isolation/enforcement perspective), there are three different pieces required :
      1) generic cgroups utilities that can be re-used across multiple resource handler ( e.g for CPU, Network, Disk). For example for net_cls we want to be able to create new cgroups, update cgroup params, read cgroup params etc.
      2) A mechanism to execute ‘PrivilegedOperation’s whose functionality requires super-user privileges and is implemented by container-executor binary
      3) Implementation that is specific to a resource type ( i.e network, disk would each have an implementation that provides isolation/enforcement for that resource type)

      Corresponding to the three pieces listed above, the patch for YARN-3443 provides the following :
      1) cgroups functionality that can be used across different resource types. CGroupsHandler.java specifies the interface and implementation is in CGroupsHandlerImpl.java . New cgroups controller types can be easily added to CGroupsHandler.java as and when necessary
      2) PrivilegedOperation.java and PrivilegedOperationExecutor.java wrap the container-executor binary and provide a way of executing operations that require elevated privileges. There are also utility functions that help ‘batching’ of certain kinds of operations in order to avoid multiple invocations of the container-executor binary
      3) ResourceHandler.java specifies an interface that custom resource handlers are expected to implement. This interface provides hooks for various operations during a container lifecyle - bootstrap, preStart, postComplete, reAcquire, teardown. Each of these hooks return a list of privileged operations - this is done so that the resulting set of privileged operations can be batched for performance reasons, if necessary. ResourceHandlerChain.java provides a simple chaining mechanism across multiple resource handlers. This is useful when multiple resource handlers are in place. They can be chained in sequence - e.g cpu, network, disk . A resource handler chain would hook in directly into LCE at various points in the container life cycle.

      1. YARN-3443.001.patch
        68 kB
        Sidharta Seethana
      2. YARN-3443.002.patch
        68 kB
        Sidharta Seethana
      3. YARN-3443.003.patch
        68 kB
        Sidharta Seethana
      4. YARN-3443.004.patch
        70 kB
        Sidharta Seethana
      5. YARN-3443.005.patch
        69 kB
        Sidharta Seethana

        Issue Links

          Activity

          Hide
          sidharta-s Sidharta Seethana added a comment -

          Attaching patch that 1) separates out CGroup implementation into a reusable class 2) creates 'PrivilegedContainerExecutor' that wraps the container-executor binary that can be used for operations that require elevated privileges 3) creates a simple ResourceHandler interface for that be used to plug in support for new resource types.

          Show
          sidharta-s Sidharta Seethana added a comment - Attaching patch that 1) separates out CGroup implementation into a reusable class 2) creates 'PrivilegedContainerExecutor' that wraps the container-executor binary that can be used for operations that require elevated privileges 3) creates a simple ResourceHandler interface for that be used to plug in support for new resource types.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12709150/YARN-3443.001.patch
          against trunk revision bad070f.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          -1 javac. The applied patch generated 1150 javac compiler warnings (more than the trunk's current 1148 warnings).

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7210//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
          Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7210//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12709150/YARN-3443.001.patch against trunk revision bad070f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 1150 javac compiler warnings (more than the trunk's current 1148 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7210//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7210//console This message is automatically generated.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Reattaching the patch with the fixed findbug warning fixed. Not sure what to make of the javac warnings here, however : https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/diffJavacWarnings.txt

          Show
          sidharta-s Sidharta Seethana added a comment - Reattaching the patch with the fixed findbug warning fixed. Not sure what to make of the javac warnings here, however : https://builds.apache.org/job/PreCommit-YARN-Build/7210//artifact/patchprocess/diffJavacWarnings.txt
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Resubmitting a new patch.

          Show
          sidharta-s Sidharta Seethana added a comment - Resubmitting a new patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12709194/YARN-3443.002.patch
          against trunk revision 72f6bd4.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          -1 javac. The applied patch generated 1150 javac compiler warnings (more than the trunk's current 1148 warnings).

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7211//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7211//artifact/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7211//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12709194/YARN-3443.002.patch against trunk revision 72f6bd4. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 1150 javac compiler warnings (more than the trunk's current 1148 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7211//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7211//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7211//console This message is automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Thanks for uploading the patch Sidharta Seethana!

          Some comments

          1. In PrivilegedOperationExecutor.java
            catch (ExitCodeException e) {
              if (LOG.isWarnEnabled()) {
                 LOG.warn("Shell execution returned exit code: " + exec.getExitCode());
                 LOG.warn("Privileged Execution Operation Output:");
                 LOG.warn(exec.getOutput());
              }
            }
            

            Can you merge the warnings into one warn message and add the exception to the log message? This is so that alerts based on the number of errors/warnings reflect the actual number.

          2. In CGroupHandlerImpl.java
            if (enableCGroupMount) {
                //nothing to do here - we support 'deferred' mounting of specific
                //controllers - we'll populate the path for a given controller when an
                //explicit mountCGroupController request is issued.
            } else {
                //cluster admins are expected to have mounted controllers in specific
                //locations - we'll attempt to figure out mount points
                initializeControllerPathsFromMtab();
            }
            

            Can we remove the empty if block?

          3. In CGroupHandlerImpl.java
            LOG.error("Mount point Based on mtab file : " + MTAB_FILE);
            LOG.error("Controller mount point not writable for: " + name);
            

            Merge the log messages

          4. In CGroupHandlerImpl.java
            LOG.warn("Failed to initialize controller paths!");
            LOG.warn(e);
            

            Merge the log messages

          5. In CGroupHandlerImpl.java
            In initializeControllerPathsFromMtab, when the error and warnings are logged, should we throw an exception, since the functionality being requested will not work?
          6. In CGroupHandlerImpl.java
            In mountCGroupController
            if (!enableCGroupMount) {
                if (LOG.isWarnEnabled()) {
                  LOG.warn("CGroup mounting is disabled - ignoring mount request for: " +
                        controller.getName());
                  return;
                }
            }
            

            Should this be an error with an exception being thrown since the functionality being requested won't be possible?

          7. In CGroupHandlerImpl.java
            @param file object referring to the cgroup to be deleted mislabelled
            

            The param name is mislabeled. It should be cgf instead of file

          8. In ResourceHandler.java
               * @param containerId if of the container being reacquired.
               * @return
               * @throws ResourceHandlerException
            

            return is missing description

          9. In ResourceHandler.java
            List<PrivilegedOperation> postComplete(ContainerId containerId) throws
                  ResourceHandlerException;
            
            ;
            

            There's an extra semi-colon.

          10. In TestPrivilegedOperationExecutor.java
            expectedPath = customExecutorPath;
            containerExePath = PrivilegedOperationExecutor
                    .getContainerExecutorExecutablePath(confWithExecutorPath);
            Assert.assertEquals(expectedPath, customExecutorPath);
            

            containerExePath is not used. The assert will always succeed since you set them to be equal before calling the function. Did you mean to compare containerExePath and expectedPath?

          11. In TestPrivilegedOperationExecutor.java
            prefixLength + 0
            

            Remove the 0?

          12. In TestPrivilegedOperationExecutor.java
            try {
                  PrivilegedOperationExecutor.squashCGroupOperations(ops);
            } catch (PrivilegedOperationException e) {
                  if (LOG.isInfoEnabled()) {
                    LOG.info("Caught expected exception : " + e);
                  }
                  caughtPrivilegedOperationException = true;
            }
            
            Assert.assertTrue("Expected squash operation to fail!",
                    caughtPrivilegedOperationException);
            

            Instead of using the boolean to check if the exception was thrown, you can use Assert.fail in the try block. This pattern occurs multiple times.

          13. I think the formatting is a bit off in a couple of files. Can you apply the formatter and check?
          Show
          vvasudev Varun Vasudev added a comment - Thanks for uploading the patch Sidharta Seethana ! Some comments In PrivilegedOperationExecutor.java catch (ExitCodeException e) { if (LOG.isWarnEnabled()) { LOG.warn("Shell execution returned exit code: " + exec.getExitCode()); LOG.warn("Privileged Execution Operation Output:"); LOG.warn(exec.getOutput()); } } Can you merge the warnings into one warn message and add the exception to the log message? This is so that alerts based on the number of errors/warnings reflect the actual number. In CGroupHandlerImpl.java if (enableCGroupMount) { //nothing to do here - we support 'deferred' mounting of specific //controllers - we'll populate the path for a given controller when an //explicit mountCGroupController request is issued. } else { //cluster admins are expected to have mounted controllers in specific //locations - we'll attempt to figure out mount points initializeControllerPathsFromMtab(); } Can we remove the empty if block? In CGroupHandlerImpl.java LOG.error("Mount point Based on mtab file : " + MTAB_FILE); LOG.error("Controller mount point not writable for: " + name); Merge the log messages In CGroupHandlerImpl.java LOG.warn("Failed to initialize controller paths!"); LOG.warn(e); Merge the log messages In CGroupHandlerImpl.java In initializeControllerPathsFromMtab, when the error and warnings are logged, should we throw an exception, since the functionality being requested will not work? In CGroupHandlerImpl.java In mountCGroupController if (!enableCGroupMount) { if (LOG.isWarnEnabled()) { LOG.warn("CGroup mounting is disabled - ignoring mount request for: " + controller.getName()); return; } } Should this be an error with an exception being thrown since the functionality being requested won't be possible? In CGroupHandlerImpl.java @param file object referring to the cgroup to be deleted mislabelled The param name is mislabeled. It should be cgf instead of file In ResourceHandler.java * @param containerId if of the container being reacquired. * @return * @throws ResourceHandlerException return is missing description In ResourceHandler.java List<PrivilegedOperation> postComplete(ContainerId containerId) throws ResourceHandlerException; ; There's an extra semi-colon. In TestPrivilegedOperationExecutor.java expectedPath = customExecutorPath; containerExePath = PrivilegedOperationExecutor .getContainerExecutorExecutablePath(confWithExecutorPath); Assert.assertEquals(expectedPath, customExecutorPath); containerExePath is not used. The assert will always succeed since you set them to be equal before calling the function. Did you mean to compare containerExePath and expectedPath? In TestPrivilegedOperationExecutor.java prefixLength + 0 Remove the 0? In TestPrivilegedOperationExecutor.java try { PrivilegedOperationExecutor.squashCGroupOperations(ops); } catch (PrivilegedOperationException e) { if (LOG.isInfoEnabled()) { LOG.info("Caught expected exception : " + e); } caughtPrivilegedOperationException = true; } Assert.assertTrue("Expected squash operation to fail!", caughtPrivilegedOperationException); Instead of using the boolean to check if the exception was thrown, you can use Assert.fail in the try block. This pattern occurs multiple times. I think the formatting is a bit off in a couple of files. Can you apply the formatter and check?
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Thanks for the review, Varun Vasudev . Responses below :
          1. I could change such log lines to use StringBuffer everywhere. However, I think a metric based on calls to LOG.warn()/LOG.error() does not accurately reflect the warn/error 'even't count.
          2. I'll add an info log line to the if block. I had added this block for better readability since the behavior here is a little different from the current implementation in CgroupsLCEResourcesHandler
          3. - 4. Same comment as 1.
          5. Sure, I can make this change.
          6. This is by design. Otherwise, every resource handler implementation that uses cgroups will have to check if cgroup mounting is enabled or not (which is error-prone). It seemed better to instead ignore a mount request when cgroup mounting is disabled.
          7. I'll fix it.
          8. I'll fix it.
          9. I'll fix it.
          10. Yikes. Yes, I'll fix it. Not sure how this one got through.
          11. I had added it for clarity, but maybe it isn't necessary. I'll remove it.
          12. I'll fix it.
          13. I applied a formatter to all file before creating the patch - but, I'll verify.

          Show
          sidharta-s Sidharta Seethana added a comment - Thanks for the review, Varun Vasudev . Responses below : 1. I could change such log lines to use StringBuffer everywhere. However, I think a metric based on calls to LOG.warn()/LOG.error() does not accurately reflect the warn/error 'even't count. 2. I'll add an info log line to the if block. I had added this block for better readability since the behavior here is a little different from the current implementation in CgroupsLCEResourcesHandler 3. - 4. Same comment as 1. 5. Sure, I can make this change. 6. This is by design. Otherwise, every resource handler implementation that uses cgroups will have to check if cgroup mounting is enabled or not (which is error-prone). It seemed better to instead ignore a mount request when cgroup mounting is disabled. 7. I'll fix it. 8. I'll fix it. 9. I'll fix it. 10. Yikes. Yes, I'll fix it. Not sure how this one got through. 11. I had added it for clarity, but maybe it isn't necessary. I'll remove it. 12. I'll fix it. 13. I applied a formatter to all file before creating the patch - but, I'll verify.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          New patch coming up, incorporating code review feedback.

          Show
          sidharta-s Sidharta Seethana added a comment - New patch coming up, incorporating code review feedback.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Patch that incorporates code review feedback from Varun Vasudev

          Show
          sidharta-s Sidharta Seethana added a comment - Patch that incorporates code review feedback from Varun Vasudev
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12723436/YARN-3443.003.patch
          against trunk revision 28bebc8.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7228//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7228//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723436/YARN-3443.003.patch against trunk revision 28bebc8. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7228//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7228//console This message is automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          Minor documentation fixes, everything else looks good.

          1. In PrivilegedOperationExecutor.java, getPrivilegedOperationExecutionCommand is documented as throwing ExitCodeException but it doesn't throw it.
          2. In CGroupsHandler.java, the documentation for createCGroup is missing descriptions for controller and path.
          Show
          vvasudev Varun Vasudev added a comment - Minor documentation fixes, everything else looks good. In PrivilegedOperationExecutor.java, getPrivilegedOperationExecutionCommand is documented as throwing ExitCodeException but it doesn't throw it. In CGroupsHandler.java, the documentation for createCGroup is missing descriptions for controller and path.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          new patch coming up.

          Show
          sidharta-s Sidharta Seethana added a comment - new patch coming up.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Patch with documentation fixes.

          Show
          sidharta-s Sidharta Seethana added a comment - Patch with documentation fixes.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12723549/YARN-3443.004.patch
          against trunk revision 3fb5abf.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7234//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7234//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723549/YARN-3443.004.patch against trunk revision 3fb5abf. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7234//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7234//console This message is automatically generated.
          Hide
          vvasudev Varun Vasudev added a comment -

          +1, lgtm for the latest patch.

          Show
          vvasudev Varun Vasudev added a comment - +1, lgtm for the latest patch.
          Hide
          djp Junping Du added a comment -

          Thanks Sidharta Seethana for delivering the patch and Varun Vasudev for reviewing.
          Just quickly walk though the patch, the patch looks good in overall. Some minor comments below, but more comments may come later:

          +        if (LOG.isWarnEnabled()) {
          +          LOG.warn("Invalid argument: " + arg);
          +        }
          

          We typically log message directly on INFO or above level (like: warn, error), not necessary for additional check for LOG.isWarnEnabled(). Also there are some other places for checking LOG.isWarnEnabled() or even LOG.isErrorEnabled() which should be unnecessary.

          +@InterfaceAudience.Private
          +@InterfaceStability.Unstable class CGroupsHandlerImpl
          +    implements CGroupsHandler {
          

          We should separate annotation from the line of calss definition. Also, it should be a public class?

          In ResourceHandler.java (and other places, like: ResourceHandlerChain)

          +  List<PrivilegedOperation> teardown() throws ResourceHandlerException;
          

          teardown => tearDown

          Show
          djp Junping Du added a comment - Thanks Sidharta Seethana for delivering the patch and Varun Vasudev for reviewing. Just quickly walk though the patch, the patch looks good in overall. Some minor comments below, but more comments may come later: + if (LOG.isWarnEnabled()) { + LOG.warn( "Invalid argument: " + arg); + } We typically log message directly on INFO or above level (like: warn, error), not necessary for additional check for LOG.isWarnEnabled(). Also there are some other places for checking LOG.isWarnEnabled() or even LOG.isErrorEnabled() which should be unnecessary. +@InterfaceAudience.Private +@InterfaceStability.Unstable class CGroupsHandlerImpl + implements CGroupsHandler { We should separate annotation from the line of calss definition. Also, it should be a public class? In ResourceHandler.java (and other places, like: ResourceHandlerChain) + List<PrivilegedOperation> teardown() throws ResourceHandlerException; teardown => tearDown
          Hide
          sidharta-s Sidharta Seethana added a comment -

          Thanks for the review, Junping Du . Responses below :
          1. I was trying to keep the 'enabled' check consistent between various log levels as a general practice. I could remove it, though.
          2. "We should separate annotation from the line of calss definition. " - this was by accident. I'll fix it. Its not a public class by design, it is meant to be used only within the resources package (Custom resource handlers are added to this package).
          3. teardown is one word, hence no camel-case

          I'll wait for further review comments from you before uploading a new patch, thanks!

          Show
          sidharta-s Sidharta Seethana added a comment - Thanks for the review, Junping Du . Responses below : 1. I was trying to keep the 'enabled' check consistent between various log levels as a general practice. I could remove it, though. 2. "We should separate annotation from the line of calss definition. " - this was by accident. I'll fix it. Its not a public class by design, it is meant to be used only within the resources package (Custom resource handlers are added to this package). 3. teardown is one word, hence no camel-case I'll wait for further review comments from you before uploading a new patch, thanks!
          Hide
          djp Junping Du added a comment -

          Thanks Sidharta Seethana for response.

          I was trying to keep the 'enabled' check consistent between various log levels as a general practice. I could remove it, though.

          The enable check on debug is reasonable because we usually turn off debug level on production for performance and have switch for check debug level can save cycles of going to log.debug. For info and above level, we usually turn on even in production environment, so checking log, warn or error is unnecessary as almost all places of current code base shows. Make sense?

          teardown is one word, hence no camel-case.

          I see. However, we have tearDown() almost everywhere in existing code base. May be we should keep consistent here? This is only an optional comments, please feel free to address or not.

          I think we may need to do some refactor work for package hierarchy here:
          so far, the interface and implementation are all put into two packages below:

          o.a.h.y.s.n.c.linux.resources.*
          o.a.h.y.s.n.c.linux.privileged.*
          

          I would prefer we separate interfaces with implementation. So linux can be seen as an implementation, other platform implementation. e.g. Windows container, can comes later. I would prefer the package hierarchy looks like something below:

          o.a.h.y.s.n.c.resources.* (for resource interface, like: ResourceHandler)
          o.a.h.y.s.n.c.resources.linux.* (for resource implementation, like: CGroupsHandlerImpl)
          o.a.h.y.s.n.c.linux.privileged.* (for privileged interface, like: PrivilegedOperation, with moving out CGroup-specific enum OperationType)
          o.a.h.y.s.n.c.privileged.linux* (for implementation details, like: PrivilegedOperationExecutor)
          
          Show
          djp Junping Du added a comment - Thanks Sidharta Seethana for response. I was trying to keep the 'enabled' check consistent between various log levels as a general practice. I could remove it, though. The enable check on debug is reasonable because we usually turn off debug level on production for performance and have switch for check debug level can save cycles of going to log.debug. For info and above level, we usually turn on even in production environment, so checking log, warn or error is unnecessary as almost all places of current code base shows. Make sense? teardown is one word, hence no camel-case. I see. However, we have tearDown() almost everywhere in existing code base. May be we should keep consistent here? This is only an optional comments, please feel free to address or not. I think we may need to do some refactor work for package hierarchy here: so far, the interface and implementation are all put into two packages below: o.a.h.y.s.n.c.linux.resources.* o.a.h.y.s.n.c.linux.privileged.* I would prefer we separate interfaces with implementation. So linux can be seen as an implementation, other platform implementation. e.g. Windows container, can comes later. I would prefer the package hierarchy looks like something below: o.a.h.y.s.n.c.resources.* ( for resource interface , like: ResourceHandler) o.a.h.y.s.n.c.resources.linux.* ( for resource implementation, like: CGroupsHandlerImpl) o.a.h.y.s.n.c.linux.privileged.* ( for privileged interface , like: PrivilegedOperation, with moving out CGroup-specific enum OperationType) o.a.h.y.s.n.c.privileged.linux* ( for implementation details, like: PrivilegedOperationExecutor)
          Hide
          sidharta-s Sidharta Seethana added a comment -

          hi Junping Du, thanks for your comments. Responses below :

          I'll remove the log level checks. About the packaging issue : this is something that was considered earlier on during the design/development. The thing is that the notion of PrivilegedOperation (and every type of PrivilegedOperation as mentioned in PrivilegedOperation.java) is linux-specific. A ResourceHandler implementation in Windows, for example, may not require this notion at all. Rather than expand the scope of this task further, it was decided that we'd keep this linux specific for now. At some point, if we choose to provide similar abstractions for Windows etc., we can decide if the same notions apply or not. Generalizing these mechanisms now, when they might not be applicable to any platform but linux, might cause more problems in the future. For now, I would prefer limiting the scope of this JIRA to linux-specific scenarios and not have to refactor this further

          Vinod Kumar Vavilapalli and Varun Vasudev - we had discussed this earlier as well. Please chime in.

          thanks

          Show
          sidharta-s Sidharta Seethana added a comment - hi Junping Du , thanks for your comments. Responses below : I'll remove the log level checks. About the packaging issue : this is something that was considered earlier on during the design/development. The thing is that the notion of PrivilegedOperation (and every type of PrivilegedOperation as mentioned in PrivilegedOperation.java) is linux-specific. A ResourceHandler implementation in Windows, for example, may not require this notion at all. Rather than expand the scope of this task further, it was decided that we'd keep this linux specific for now. At some point, if we choose to provide similar abstractions for Windows etc., we can decide if the same notions apply or not. Generalizing these mechanisms now, when they might not be applicable to any platform but linux, might cause more problems in the future. For now, I would prefer limiting the scope of this JIRA to linux-specific scenarios and not have to refactor this further Vinod Kumar Vavilapalli and Varun Vasudev - we had discussed this earlier as well. Please chime in. thanks
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          I would prefer we separate interfaces with implementation.

          I've tried to get this done this way offline, but we unfortunately came to the conclusion that this may not be straightforward given the tight coupling between LCE, CgroupsHandler, container-executor binary etc. I think this is a reasonable first cut at cleaning up things for Linux, with the goal of adding more resource types like disk to Linux less painful.

          Existing resource-types added to Windows (YARN-2190) are already laid out in some fashion, I'd not do another refactor before carefully looking at how to structure them together. Let's keep the focus of this JIRA to extend resources for Linux and branch the cross-platform refactoring off to its own JIRA.

          IAC, let's make sure that none of this internal packaging structure leaks through to the end-users via configuration property names or documentation.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - I would prefer we separate interfaces with implementation. I've tried to get this done this way offline, but we unfortunately came to the conclusion that this may not be straightforward given the tight coupling between LCE, CgroupsHandler, container-executor binary etc. I think this is a reasonable first cut at cleaning up things for Linux, with the goal of adding more resource types like disk to Linux less painful. Existing resource-types added to Windows ( YARN-2190 ) are already laid out in some fashion, I'd not do another refactor before carefully looking at how to structure them together. Let's keep the focus of this JIRA to extend resources for Linux and branch the cross-platform refactoring off to its own JIRA. IAC, let's make sure that none of this internal packaging structure leaks through to the end-users via configuration property names or documentation.
          Hide
          djp Junping Du added a comment -

          Thanks Vinod Kumar Vavilapalli for comments and confirmation on this. I just want to make sure we don't miss other platforms/implementations in our previous discussions/design. I am OK with keep current package structure for now - if needed, we can do more refactor later when Windows is ready in future. Sidharta Seethana, would you address my other comments in new patch? Thx!

          Show
          djp Junping Du added a comment - Thanks Vinod Kumar Vavilapalli for comments and confirmation on this. I just want to make sure we don't miss other platforms/implementations in our previous discussions/design. I am OK with keep current package structure for now - if needed, we can do more refactor later when Windows is ready in future. Sidharta Seethana , would you address my other comments in new patch? Thx!
          Hide
          sidharta-s Sidharta Seethana added a comment -

          hi Junping Du , uploading a new patch. Please take a look.

          thanks.

          Show
          sidharta-s Sidharta Seethana added a comment - hi Junping Du , uploading a new patch. Please take a look. thanks.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12725010/YARN-3443.005.patch
          against trunk revision 7fc50e2.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 43 warning messages.
          See https://builds.apache.org/job/PreCommit-YARN-Build/7318//artifact/patchprocess/diffJavadocWarnings.txt for details.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7318//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7318//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12725010/YARN-3443.005.patch against trunk revision 7fc50e2. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 43 warning messages. See https://builds.apache.org/job/PreCommit-YARN-Build/7318//artifact/patchprocess/diffJavadocWarnings.txt for details. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7318//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7318//console This message is automatically generated.
          Hide
          sidharta-s Sidharta Seethana added a comment -

          The javadoc warnings are unrelated to this patch - I don't find any mention of the classes included in the patch.

          Show
          sidharta-s Sidharta Seethana added a comment - The javadoc warnings are unrelated to this patch - I don't find any mention of the classes included in the patch.
          Hide
          djp Junping Du added a comment -

          +1. Latest patch LGTM. Double check all javadoc warnings under hadoop-yarn-server-nodemanager component, nothing is related to this patch (all under hadoop-yarn-server-nodemanager).
          Will commit it shortly.

          Show
          djp Junping Du added a comment - +1. Latest patch LGTM. Double check all javadoc warnings under hadoop-yarn-server-nodemanager component, nothing is related to this patch (all under hadoop-yarn-server-nodemanager). Will commit it shortly.
          Hide
          djp Junping Du added a comment -

          I have commit 005 patch to trunk and branch-2.
          Thanks Sidharta Seethana for contributing the patch and review comments from Varun Vasudev and Vinod Kumar Vavilapalli!

          Show
          djp Junping Du added a comment - I have commit 005 patch to trunk and branch-2. Thanks Sidharta Seethana for contributing the patch and review comments from Varun Vasudev and Vinod Kumar Vavilapalli !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7579 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7579/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7579 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7579/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/163/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #163 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/163/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2095 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2095/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2095 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2095/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #897 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/897/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #897 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/897/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #154 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/154/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #154 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/154/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/164/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/164/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2113 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2113/)
          YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2113 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2113/ ) YARN-3443 . Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. (junping_du: rev 838b06ac87339494cea706d4a97e7f5383bdc442) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/TestPrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerException.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperation.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/ResourceHandlerChain.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/privileged/PrivilegedOperationException.java

            People

            • Assignee:
              sidharta-s Sidharta Seethana
              Reporter:
              sidharta-s Sidharta Seethana
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development