Details

    • Type: Sub-task
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:
    • Target Version/s:

      Description

      The RM needs to (optionally) integrate with the YARN registry:

      1. startup: create the /services and /users paths with system ACLs (yarn, hdfs principals)
      2. app-launch: create the user directory /users/$username with the relevant permissions (CRD) for them to create subnodes.
      3. attempt, container, app completion: remove service records with the matching persistence and ID
      1. YARN-2571-001.patch
        49 kB
        Steve Loughran
      2. YARN-2571-002.patch
        40 kB
        Steve Loughran
      3. YARN-2571-003.patch
        43 kB
        Steve Loughran
      4. YARN-2571-005.patch
        48 kB
        Steve Loughran
      5. YARN-2571-007.patch
        48 kB
        Steve Loughran
      6. YARN-2571-008.patch
        48 kB
        Steve Loughran
      7. YARN-2571-009.patch
        53 kB
        Steve Loughran
      8. YARN-2571-010.patch
        52 kB
        Steve Loughran
      9. YARN-2571-012.patch
        52 kB
        Steve Loughran
      10. YARN-2571-013.patch
        136 kB
        Steve Loughran
      11. YARN-2571-015.patch
        176 kB
        Steve Loughran
      12. YARN-2571-016.patch
        202 kB
        Steve Loughran
      13. YARN-2571-017.patch
        201 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          stevel@apache.org Steve Loughran added a comment -

          This is the patch of everything under hadoop-yarn-server to integrate the registry with the RM. The RM takes on the tasks of

          1. creating user nodes with their access permissions (and the system accounts)
          2. recursively purging records whose persistence is tied to container, app-attempt or app when they terminate
          Show
          stevel@apache.org Steve Loughran added a comment - This is the patch of everything under hadoop-yarn-server to integrate the registry with the RM. The RM takes on the tasks of creating user nodes with their access permissions (and the system accounts) recursively purging records whose persistence is tied to container, app-attempt or app when they terminate
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch atop the (now applied) YARN-2562 changes.

          Most of this patch is simply extending the RMContextImpl constructors in tests; RMRegistryService and RMRegistryOperationsService the new classes.

          The tests for this code are in the Distributed Shell patch: if we want an integrated code+test patch, YARN-2646 should be included here

          Show
          stevel@apache.org Steve Loughran added a comment - patch atop the (now applied) YARN-2562 changes. Most of this patch is simply extending the RMContextImpl constructors in tests; RMRegistryService and RMRegistryOperationsService the new classes. The tests for this code are in the Distributed Shell patch: if we want an integrated code+test patch, YARN-2646 should be included here
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12673705/YARN-2571-002.patch
          against trunk revision db7f165.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5366//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/5366//artifact/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5366//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12673705/YARN-2571-002.patch against trunk revision db7f165. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 15 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 1 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5366//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/5366//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5366//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch -003 (against branch-2). Exceptions thrown in the async future actions are logged @ warn level before being rethrown. As the RM doesn't issue future.get() calls those exceptions do not propagate to the RM (i.e. registry problems do not stop the RM working, simply registry setup/teardown). The logging ensures that these problems are logged.

          Show
          stevel@apache.org Steve Loughran added a comment - patch -003 (against branch-2). Exceptions thrown in the async future actions are logged @ warn level before being rethrown. As the RM doesn't issue future.get() calls those exceptions do not propagate to the RM (i.e. registry problems do not stop the RM working, simply registry setup/teardown). The logging ensures that these problems are logged.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12674613/YARN-2571-003.patch
          against trunk revision 178bc50.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5382//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5382//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674613/YARN-2571-003.patch against trunk revision 178bc50. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 15 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5382//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5382//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 005.

          1. in sync with trunk rev 0942c9
          2. moved the path creation logic in the RegistryAdminService.start() operation to being async
          3. tests enhanced to await completion of async setup.

          The reason to move to async is to eliminate the impact of ZK connection problems on RM startup. With this patch all RM->registry operations are executed in a single executor thread, which implicitly queues the requests in an ordered sequence.

          There is one side-effect: if there is a failure in registry startup due to security problems, this is not propagated to the RM —i.e. it does not cause RM startup to fail. It will however be visible to users of the registry who themselves are likely to have auth problems.

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 005. in sync with trunk rev 0942c9 moved the path creation logic in the RegistryAdminService.start() operation to being async tests enhanced to await completion of async setup. The reason to move to async is to eliminate the impact of ZK connection problems on RM startup. With this patch all RM->registry operations are executed in a single executor thread, which implicitly queues the requests in an ordered sequence. There is one side-effect: if there is a failure in registry startup due to security problems, this is not propagated to the RM —i.e. it does not cause RM startup to fail. It will however be visible to users of the registry who themselves are likely to have auth problems.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12676886/YARN-2571-005.patch
          against trunk revision 0942c99.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5541//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5541//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676886/YARN-2571-005.patch against trunk revision 0942c99. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5541//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5541//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 006: upgrade "use a URI in the api" from a SHOULD to a MUST; fix examples and tests accordingly

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 006: upgrade "use a URI in the api" from a SHOULD to a MUST; fix examples and tests accordingly
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 007; rebased onto trunk and purges unused event handlers

          Show
          stevel@apache.org Steve Loughran added a comment - patch 007; rebased onto trunk and purges unused event handlers
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 008

          fixes a bug noted by Jian He: the event handler for app attempts was invoking the AM container cleanup and so prematurely deleting records tagged as persistent to app attempts.

          This patch maps it to container completion events.

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 008 fixes a bug noted by Jian He: the event handler for app attempts was invoking the AM container cleanup and so prematurely deleting records tagged as persistent to app attempts. This patch maps it to container completion events.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12678960/YARN-2571-008.patch
          against trunk revision 67f13b5.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5698//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5698//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12678960/YARN-2571-008.patch against trunk revision 67f13b5. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5698//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5698//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch -009 in sync with trunk

          Show
          stevel@apache.org Steve Loughran added a comment - patch -009 in sync with trunk
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
          against trunk revision 82707b4.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6056//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6056//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch against trunk revision 82707b4. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6056//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6056//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
          against trunk revision 82707b4.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6057//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6057//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch against trunk revision 82707b4. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6057//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6057//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
          against trunk revision 4cd66f7.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6254//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6254//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6254//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch against trunk revision 4cd66f7. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6254//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6254//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6254//console This message is automatically generated.
          Hide
          xgong Xuan Gong added a comment -

          Thanks for the patch. Overall looks fine.

          1. Could we move the registry service from alway-on service to active services ? For example, if RM HA is enabled, only the active RM can start the registry service.

          2. The Curator Framework is used here to do the ZK operation. I am not familiar with this. Does the curator framework provide automatic fence mechanism when we write/delete related znodes? For example, only the active RM can write data in znode. The standby RM should not be allowed to do anything.

          Show
          xgong Xuan Gong added a comment - Thanks for the patch. Overall looks fine. 1. Could we move the registry service from alway-on service to active services ? For example, if RM HA is enabled, only the active RM can start the registry service. 2. The Curator Framework is used here to do the ZK operation. I am not familiar with this. Does the curator framework provide automatic fence mechanism when we write/delete related znodes? For example, only the active RM can write data in znode. The standby RM should not be allowed to do anything.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Apologies for coming in really late on this.

          startup: create the /services and /users paths with system ACLs (yarn, hdfs principals)

          app-launch: create the user directory /users/$username with the relevant permissions (CRD) for them to create subnodes.

          None of this is RM responsibility. Similar to creation of user directories on HDFS, this needs to be taken care of by administrators/external systems.

          bq. attempt, container, app completion: remove service records with the matching persistence and ID

          This looks like application-level responsibility. Removing records on container-completion can and should be done by the individual apps' ApplicationMasters. Removing records on app completion should be done in an application-cleanup container (YARN-2261). Any use-case for application-attempt level records?

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Apologies for coming in really late on this. startup: create the /services and /users paths with system ACLs (yarn, hdfs principals) app-launch: create the user directory /users/$username with the relevant permissions (CRD) for them to create subnodes. None of this is RM responsibility. Similar to creation of user directories on HDFS, this needs to be taken care of by administrators/external systems. bq. attempt, container, app completion: remove service records with the matching persistence and ID This looks like application-level responsibility. Removing records on container-completion can and should be done by the individual apps' ApplicationMasters. Removing records on app completion should be done in an application-cleanup container ( YARN-2261 ). Any use-case for application-attempt level records?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Vinod,

          having the RM create the user paths allows the registry to be set up with the correct permissions as YARN jobs are created. Without that, if there is no path for that user set up, the application is likely to fail post-launch with some error.

          For cleanup, automatic purging of records keeps the registry data somewhat under control, without applications having to go to the effort of writing these not-yet-implemented cleanup containers. It's not a particularly complex piece of code; there's tests for the distributed shell that verify it works in YARN-2646

          Show
          stevel@apache.org Steve Loughran added a comment - Vinod, having the RM create the user paths allows the registry to be set up with the correct permissions as YARN jobs are created. Without that, if there is no path for that user set up, the application is likely to fail post-launch with some error. For cleanup, automatic purging of records keeps the registry data somewhat under control, without applications having to go to the effort of writing these not-yet-implemented cleanup containers. It's not a particularly complex piece of code; there's tests for the distributed shell that verify it works in YARN-2646
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Xuan

          1. If you can show me an example of an active service to start with, I'll gladly make it active only

          2. We're relying on operations to be idempotent: whoever creates last wins, whoever deletes last wins. There's some race conditions on cleanup if there's a change between a read and a delete, but that's what you get in a world without transactions.

          Show
          stevel@apache.org Steve Loughran added a comment - Xuan 1. If you can show me an example of an active service to start with, I'll gladly make it active only 2. We're relying on operations to be idempotent: whoever creates last wins, whoever deletes last wins. There's some race conditions on cleanup if there's a change between a read and a delete, but that's what you get in a world without transactions.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          in sync with trunk and moved registry creation to createAndInitActiveServices()

          Show
          stevel@apache.org Steve Loughran added a comment - in sync with trunk and moved registry creation to createAndInitActiveServices()
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch
          against trunk revision e0ec071.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6575//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6575//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch against trunk revision e0ec071. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 16 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6575//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6575//console This message is automatically generated.
          Hide
          srikanth.sampath Srikanth Sampath added a comment -

          What's the status of this patch - Steve Loughran I am considering using YARN registry for MR AppMaster in YARN-3972 and want to take some learnings from here.

          Show
          srikanth.sampath Srikanth Sampath added a comment - What's the status of this patch - Steve Loughran I am considering using YARN registry for MR AppMaster in YARN-3972 and want to take some learnings from here.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / eee0d45
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/8905/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 Console output https://builds.apache.org/job/PreCommit-YARN-Build/8905/console This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          All the registry code is there (slider uses it), except two bits of the RM side

          1. create the base user path on app launch (in case the app needs it). This needs to be done by a process with the right permissions on ZK; it also makes sure that the user path is created with the perms to allow the RM/admin to delete it
          2. purge entries on container/AM failure

          There was push-back from the YARN team on #2; not for the RM. I do still think #2 is needed. Irrespective of that, there is a main() entry point in the 2.7+ code which offers a CLI to create the reg; its just without docs or tests. Email me direct if you want to start using the code & I'll help you

          Show
          stevel@apache.org Steve Loughran added a comment - All the registry code is there (slider uses it), except two bits of the RM side create the base user path on app launch (in case the app needs it). This needs to be done by a process with the right permissions on ZK; it also makes sure that the user path is created with the perms to allow the RM/admin to delete it purge entries on container/AM failure There was push-back from the YARN team on #2; not for the RM. I do still think #2 is needed. Irrespective of that, there is a main() entry point in the 2.7+ code which offers a CLI to create the reg; its just without docs or tests. Email me direct if you want to start using the code & I'll help you
          Hide
          srikanth.sampath Srikanth Sampath added a comment -

          Thanks Steve Loughran. I will ping you directly.

          Show
          srikanth.sampath Srikanth Sampath added a comment - Thanks Steve Loughran . I will ping you directly.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user steveloughran opened a pull request:

          https://github.com/apache/hadoop/pull/66

          YARN-2571 RM to support YARN registry

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/steveloughran/hadoop YARN-913/YARN-2571-RM-on-trunk

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/66.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #66


          commit 25a56da6037bb50d3ce5bbcc3001914e51ea2457
          Author: Steve Loughran <stevel@hortonworks.com>
          Date: 2015-11-11T20:20:23Z

          YARN-2571 RM setup of registry, reapplied to trunk


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user steveloughran opened a pull request: https://github.com/apache/hadoop/pull/66 YARN-2571 RM to support YARN registry You can merge this pull request into a Git repository by running: $ git pull https://github.com/steveloughran/hadoop YARN-913 / YARN-2571 -RM-on-trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/66.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #66 commit 25a56da6037bb50d3ce5bbcc3001914e51ea2457 Author: Steve Loughran <stevel@hortonworks.com> Date: 2015-11-11T20:20:23Z YARN-2571 RM setup of registry, reapplied to trunk
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 13 new or modified test files.
          +1 mvninstall 8m 18s trunk passed
          +1 compile 2m 10s trunk passed with JDK v1.8.0_66
          +1 compile 2m 22s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 44s trunk passed
          +1 findbugs 3m 6s trunk passed
          -1 javadoc 0m 33s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66.
          +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91
          -1 mvninstall 0m 36s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 compile 2m 16s hadoop-yarn in the patch failed with JDK v1.8.0_66.
          -1 javac 2m 16s hadoop-yarn in the patch failed with JDK v1.8.0_66.
          -1 compile 2m 12s hadoop-yarn in the patch failed with JDK v1.7.0_91.
          -1 javac 2m 12s hadoop-yarn in the patch failed with JDK v1.7.0_91.
          -1 checkstyle 0m 37s Patch generated 50 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 140, now 184).
          -1 mvnsite 0m 38s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 mvneclipse 0m 47s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          -1 findbugs 0m 37s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 javadoc 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 javadoc 0m 15s hadoop-yarn-registry in the patch failed with JDK v1.8.0_66.
          +1 javadoc 1m 15s the patch passed with JDK v1.7.0_91
          +1 unit 0m 29s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_66.
          -1 unit 0m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 15s hadoop-yarn-registry in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 32s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_91.
          -1 unit 0m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 unit 1m 18s hadoop-yarn-registry in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          41m 55s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 3d440bd23fb2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 38c4c14
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10209/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10209/console
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10209/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 13 new or modified test files. +1 mvninstall 8m 18s trunk passed +1 compile 2m 10s trunk passed with JDK v1.8.0_66 +1 compile 2m 22s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 3m 6s trunk passed -1 javadoc 0m 33s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91 -1 mvninstall 0m 36s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 2m 16s hadoop-yarn in the patch failed with JDK v1.8.0_66. -1 javac 2m 16s hadoop-yarn in the patch failed with JDK v1.8.0_66. -1 compile 2m 12s hadoop-yarn in the patch failed with JDK v1.7.0_91. -1 javac 2m 12s hadoop-yarn in the patch failed with JDK v1.7.0_91. -1 checkstyle 0m 37s Patch generated 50 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 140, now 184). -1 mvnsite 0m 38s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 47s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. -1 findbugs 0m 37s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 javadoc 0m 15s hadoop-yarn-registry in the patch failed with JDK v1.8.0_66. +1 javadoc 1m 15s the patch passed with JDK v1.7.0_91 +1 unit 0m 29s hadoop-yarn-server-common in the patch passed with JDK v1.8.0_66. -1 unit 0m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 unit 1m 15s hadoop-yarn-registry in the patch passed with JDK v1.8.0_66. +1 unit 0m 32s hadoop-yarn-server-common in the patch passed with JDK v1.7.0_91. -1 unit 0m 35s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 unit 1m 18s hadoop-yarn-registry in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 41m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 3d440bd23fb2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 38c4c14 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10209/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10209/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10209/console Console output https://builds.apache.org/job/PreCommit-YARN-Build/10209/console This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 012: rebase to trunk

          This patch is coming up to second birthday. Can someone pleae look at at it. Thanks

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 012: rebase to trunk This patch is coming up to second birthday. Can someone pleae look at at it. Thanks
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 13 new or modified test files.
          0 mvndep 0m 48s Maven dependency ordering for branch
          +1 mvninstall 6m 51s trunk passed
          +1 compile 2m 17s trunk passed
          +1 checkstyle 0m 46s trunk passed
          +1 mvnsite 1m 18s trunk passed
          +1 mvneclipse 0m 43s trunk passed
          +1 findbugs 1m 56s trunk passed
          +1 javadoc 0m 50s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          -1 mvninstall 0m 28s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 compile 1m 39s hadoop-yarn in the patch failed.
          -1 javac 1m 39s hadoop-yarn in the patch failed.
          -1 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 696 unchanged - 10 fixed = 748 total (was 706)
          -1 mvnsite 0m 29s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 mvneclipse 0m 35s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          -1 findbugs 0m 23s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 javadoc 0m 11s hadoop-yarn-registry in the patch failed.
          +1 unit 0m 49s hadoop-yarn-registry in the patch passed.
          +1 unit 0m 25s hadoop-yarn-server-common in the patch passed.
          -1 unit 0m 28s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          26m 17s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux b4ae3cd62fff 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cc01ed70
          Default Java 1.8.0_101
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13082/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13082/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 13 new or modified test files. 0 mvndep 0m 48s Maven dependency ordering for branch +1 mvninstall 6m 51s trunk passed +1 compile 2m 17s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 18s trunk passed +1 mvneclipse 0m 43s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 50s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch -1 mvninstall 0m 28s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 1m 39s hadoop-yarn in the patch failed. -1 javac 1m 39s hadoop-yarn in the patch failed. -1 checkstyle 0m 46s hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 696 unchanged - 10 fixed = 748 total (was 706) -1 mvnsite 0m 29s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 35s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 2s The patch has no ill-formed XML file. -1 findbugs 0m 23s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 11s hadoop-yarn-registry in the patch failed. +1 unit 0m 49s hadoop-yarn-registry in the patch passed. +1 unit 0m 25s hadoop-yarn-server-common in the patch passed. -1 unit 0m 28s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 26m 17s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b4ae3cd62fff 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cc01ed70 Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13082/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13082/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13082/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 013

          • fix compile problem against new yarn test
          • do as much as possible to shut up checkstyle and javadoc
          Show
          stevel@apache.org Steve Loughran added a comment - Patch 013 fix compile problem against new yarn test do as much as possible to shut up checkstyle and javadoc
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 13 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 6m 39s trunk passed
          +1 compile 2m 17s trunk passed
          +1 checkstyle 0m 46s trunk passed
          +1 mvnsite 1m 26s trunk passed
          +1 mvneclipse 0m 47s trunk passed
          +1 findbugs 2m 19s trunk passed
          +1 javadoc 0m 53s trunk passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          -1 mvninstall 0m 34s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 compile 1m 59s hadoop-yarn in the patch failed.
          -1 javac 1m 59s hadoop-yarn in the patch failed.
          -1 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 695 unchanged - 10 fixed = 747 total (was 705)
          -1 mvnsite 0m 35s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 mvneclipse 0m 39s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          -1 findbugs 0m 27s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 javadoc 0m 12s hadoop-yarn-registry in the patch failed.
          +1 unit 0m 47s hadoop-yarn-registry in the patch passed.
          +1 unit 0m 26s hadoop-yarn-server-common in the patch passed.
          -1 unit 0m 29s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          27m 13s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 4e62ed6fc545 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cc01ed70
          Default Java 1.8.0_101
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13084/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13084/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 13 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 2m 17s trunk passed +1 checkstyle 0m 46s trunk passed +1 mvnsite 1m 26s trunk passed +1 mvneclipse 0m 47s trunk passed +1 findbugs 2m 19s trunk passed +1 javadoc 0m 53s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch -1 mvninstall 0m 34s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 1m 59s hadoop-yarn in the patch failed. -1 javac 1m 59s hadoop-yarn in the patch failed. -1 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 695 unchanged - 10 fixed = 747 total (was 705) -1 mvnsite 0m 35s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 39s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 2s The patch has no ill-formed XML file. -1 findbugs 0m 27s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 12s hadoop-yarn-registry in the patch failed. +1 unit 0m 47s hadoop-yarn-registry in the patch passed. +1 unit 0m 26s hadoop-yarn-server-common in the patch passed. -1 unit 0m 29s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 27m 13s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 4e62ed6fc545 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cc01ed70 Default Java 1.8.0_101 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13084/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13084/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13084/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          YARN-2517 patch 015: remove a test which was obsolete, but which didn't show as it was broken. Fix up more javadocs

          Show
          stevel@apache.org Steve Loughran added a comment - YARN-2517 patch 015: remove a test which was obsolete, but which didn't show as it was broken. Fix up more javadocs
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 6m 49s trunk passed
          +1 compile 2m 20s trunk passed
          +1 checkstyle 0m 50s trunk passed
          +1 mvnsite 1m 41s trunk passed
          +1 mvneclipse 0m 53s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
          +1 findbugs 2m 18s trunk passed
          +1 javadoc 1m 4s trunk passed
          0 mvndep 0m 10s Maven dependency ordering for patch
          +1 mvninstall 1m 19s the patch passed
          +1 compile 2m 47s the patch passed
          -1 javac 2m 47s hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35)
          -1 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 788 unchanged - 168 fixed = 803 total (was 956)
          +1 mvnsite 1m 40s the patch passed
          +1 mvneclipse 0m 51s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 whitespace 0m 0s The patch 4 line(s) with tabs.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
          +1 findbugs 2m 32s the patch passed
          +1 javadoc 0m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52)
          +1 javadoc 0m 13s hadoop-yarn-server-common in the patch passed.
          +1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 javadoc 0m 8s hadoop-yarn-site in the patch passed.
          +1 unit 0m 48s hadoop-yarn-registry in the patch passed.
          +1 unit 0m 26s hadoop-yarn-server-common in the patch passed.
          +1 unit 37m 36s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 unit 0m 7s hadoop-yarn-site in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          67m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 8d6e2cd8e6db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9faccd1
          Default Java 1.8.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-tabs.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13086/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13086/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 49s trunk passed +1 compile 2m 20s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 41s trunk passed +1 mvneclipse 0m 53s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 18s trunk passed +1 javadoc 1m 4s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 19s the patch passed +1 compile 2m 47s the patch passed -1 javac 2m 47s hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35) -1 checkstyle 0m 49s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 788 unchanged - 168 fixed = 803 total (was 956) +1 mvnsite 1m 40s the patch passed +1 mvneclipse 0m 51s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 0s The patch 4 line(s) with tabs. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 32s the patch passed +1 javadoc 0m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52) +1 javadoc 0m 13s hadoop-yarn-server-common in the patch passed. +1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch passed. +1 javadoc 0m 8s hadoop-yarn-site in the patch passed. +1 unit 0m 48s hadoop-yarn-registry in the patch passed. +1 unit 0m 26s hadoop-yarn-server-common in the patch passed. +1 unit 37m 36s hadoop-yarn-server-resourcemanager in the patch passed. +1 unit 0m 7s hadoop-yarn-site in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 67m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 8d6e2cd8e6db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9faccd1 Default Java 1.8.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-tabs.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13086/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13086/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch 016, fixup of final javadoc warnings, cleanup of <String> style definitions, and make sure public/private/evolving scope is everywhere appropriate, including package infos

          Show
          stevel@apache.org Steve Loughran added a comment - patch 016, fixup of final javadoc warnings, cleanup of <String> style definitions, and make sure public/private/evolving scope is everywhere appropriate, including package infos
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 6m 40s trunk passed
          +1 compile 2m 16s trunk passed
          +1 checkstyle 0m 50s trunk passed
          +1 mvnsite 1m 30s trunk passed
          +1 mvneclipse 0m 51s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
          +1 findbugs 1m 53s trunk passed
          +1 javadoc 0m 57s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 8s the patch passed
          +1 compile 2m 13s the patch passed
          -1 javac 2m 13s hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35)
          -1 checkstyle 0m 47s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 787 unchanged - 168 fixed = 802 total (was 955)
          +1 mvnsite 1m 22s the patch passed
          +1 mvneclipse 0m 44s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 whitespace 0m 0s The patch 4 line(s) with tabs.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
          +1 findbugs 2m 11s the patch passed
          +1 javadoc 0m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52)
          +1 javadoc 0m 13s hadoop-yarn-server-common in the patch passed.
          +1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 javadoc 0m 7s hadoop-yarn-site in the patch passed.
          +1 unit 0m 41s hadoop-yarn-registry in the patch passed.
          +1 unit 0m 26s hadoop-yarn-server-common in the patch passed.
          -1 unit 33m 37s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 unit 0m 7s hadoop-yarn-site in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          61m 3s



          Reason Tests
          Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 41f4c09dc8d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9faccd1
          Default Java 1.8.0_101
          findbugs v3.0.0
          javac https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13089/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13089/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 6m 40s trunk passed +1 compile 2m 16s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 30s trunk passed +1 mvneclipse 0m 51s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 1m 53s trunk passed +1 javadoc 0m 57s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 8s the patch passed +1 compile 2m 13s the patch passed -1 javac 2m 13s hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35) -1 checkstyle 0m 47s hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 787 unchanged - 168 fixed = 802 total (was 955) +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 44s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 0s The patch 4 line(s) with tabs. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site +1 findbugs 2m 11s the patch passed +1 javadoc 0m 11s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52) +1 javadoc 0m 13s hadoop-yarn-server-common in the patch passed. +1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch passed. +1 javadoc 0m 7s hadoop-yarn-site in the patch passed. +1 unit 0m 41s hadoop-yarn-registry in the patch passed. +1 unit 0m 26s hadoop-yarn-server-common in the patch passed. -1 unit 33m 37s hadoop-yarn-server-resourcemanager in the patch failed. +1 unit 0m 7s hadoop-yarn-site in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 61m 3s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 41f4c09dc8d5 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9faccd1 Default Java 1.8.0_101 findbugs v3.0.0 javac https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13089/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13089/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13089/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch 017, sync up with trunk

          Can I observe that this patch is approaching its third birthday. I would really like to see it in, so that you can use the registry in a secure cluster. I'm happy to accept feedback and rework it, but we need to get this in.

          I need reviewers here. Junping Du, are you able to look at it?

          Show
          stevel@apache.org Steve Loughran added a comment - Patch 017, sync up with trunk Can I observe that this patch is approaching its third birthday. I would really like to see it in, so that you can use the registry in a secure cluster. I'm happy to accept feedback and rework it, but we need to get this in. I need reviewers here. Junping Du , are you able to look at it?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 5s YARN-2571 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue YARN-2571
          GITHUB PR https://github.com/apache/hadoop/pull/66
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/16381/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s YARN-2571 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue YARN-2571 GITHUB PR https://github.com/apache/hadoop/pull/66 Console output https://builds.apache.org/job/PreCommit-YARN-Build/16381/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Sure. Steve Loughran. I can help here.
          It sounds like the patch cannot apply to trunk again, can you rebase to current trunk? Thx!

          Show
          djp Junping Du added a comment - Sure. Steve Loughran . I can help here. It sounds like the patch cannot apply to trunk again, can you rebase to current trunk? Thx!

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated:

                Development