Details
-
Type:
Sub-task
-
Status: Patch Available
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 2.7.3
-
Fix Version/s: None
-
Component/s: resourcemanager
-
Labels:
-
Target Version/s:
Description
The RM needs to (optionally) integrate with the YARN registry:
- startup: create the /services and /users paths with system ACLs (yarn, hdfs principals)
- app-launch: create the user directory /users/$username with the relevant permissions (CRD) for them to create subnodes.
- attempt, container, app completion: remove service records with the matching persistence and ID
Issue Links
- depends upon
-
YARN-2652
add hadoop-yarn-registry package under hadoop-yarn
-
- Closed
-
- is depended upon by
-
SLIDER-552
remove root registry path setup in AM
-
- Open
-
-
YARN-2646
distributed shell & tests to use registry
-
- Open
-
-
SLIDER-149
Support a YARN service registry
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
patch atop the (now applied) YARN-2562 changes.
Most of this patch is simply extending the RMContextImpl constructors in tests; RMRegistryService and RMRegistryOperationsService the new classes.
The tests for this code are in the Distributed Shell patch: if we want an integrated code+test patch, YARN-2646 should be included here
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12673705/YARN-2571-002.patch
against trunk revision db7f165.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
-1 release audit. The applied patch generated 1 release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5366//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-YARN-Build/5366//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5366//console
This message is automatically generated.
patch -003 (against branch-2). Exceptions thrown in the async future actions are logged @ warn level before being rethrown. As the RM doesn't issue future.get() calls those exceptions do not propagate to the RM (i.e. registry problems do not stop the RM working, simply registry setup/teardown). The logging ensures that these problems are logged.
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12674613/YARN-2571-003.patch
against trunk revision 178bc50.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 15 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5382//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5382//console
This message is automatically generated.
Patch 005.
- in sync with trunk rev 0942c9
- moved the path creation logic in the RegistryAdminService.start() operation to being async
- tests enhanced to await completion of async setup.
The reason to move to async is to eliminate the impact of ZK connection problems on RM startup. With this patch all RM->registry operations are executed in a single executor thread, which implicitly queues the requests in an ordered sequence.
There is one side-effect: if there is a failure in registry startup due to security problems, this is not propagated to the RM —i.e. it does not cause RM startup to fail. It will however be visible to users of the registry who themselves are likely to have auth problems.
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12676886/YARN-2571-005.patch
against trunk revision 0942c99.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5541//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5541//console
This message is automatically generated.
Patch 006: upgrade "use a URI in the api" from a SHOULD to a MUST; fix examples and tests accordingly
patch 007; rebased onto trunk and purges unused event handlers
Patch 008
fixes a bug noted by Jian He: the event handler for app attempts was invoking the AM container cleanup and so prematurely deleting records tagged as persistent to app attempts.
This patch maps it to container completion events.
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12678960/YARN-2571-008.patch
against trunk revision 67f13b5.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5698//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5698//console
This message is automatically generated.
patch -009 in sync with trunk
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
against trunk revision 82707b4.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6056//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6056//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
against trunk revision 82707b4.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6057//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6057//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12685837/YARN-2571-009.patch
against trunk revision 4cd66f7.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6254//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6254//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6254//console
This message is automatically generated.
Thanks for the patch. Overall looks fine.
1. Could we move the registry service from alway-on service to active services ? For example, if RM HA is enabled, only the active RM can start the registry service.
2. The Curator Framework is used here to do the ZK operation. I am not familiar with this. Does the curator framework provide automatic fence mechanism when we write/delete related znodes? For example, only the active RM can write data in znode. The standby RM should not be allowed to do anything.
Apologies for coming in really late on this.
startup: create the /services and /users paths with system ACLs (yarn, hdfs principals)
app-launch: create the user directory /users/$username with the relevant permissions (CRD) for them to create subnodes.
None of this is RM responsibility. Similar to creation of user directories on HDFS, this needs to be taken care of by administrators/external systems.
bq. attempt, container, app completion: remove service records with the matching persistence and ID
This looks like application-level responsibility. Removing records on container-completion can and should be done by the individual apps' ApplicationMasters. Removing records on app completion should be done in an application-cleanup container (YARN-2261). Any use-case for application-attempt level records?
Vinod,
having the RM create the user paths allows the registry to be set up with the correct permissions as YARN jobs are created. Without that, if there is no path for that user set up, the application is likely to fail post-launch with some error.
For cleanup, automatic purging of records keeps the registry data somewhat under control, without applications having to go to the effort of writing these not-yet-implemented cleanup containers. It's not a particularly complex piece of code; there's tests for the distributed shell that verify it works in YARN-2646
Xuan
1. If you can show me an example of an active service to start with, I'll gladly make it active only
2. We're relying on operations to be idempotent: whoever creates last wins, whoever deletes last wins. There's some race conditions on cleanup if there's a change between a read and a delete, but that's what you get in a world without transactions.
in sync with trunk and moved registry creation to createAndInitActiveServices()
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch
against trunk revision e0ec071.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 16 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6575//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6575//console
This message is automatically generated.
What's the status of this patch - Steve Loughran I am considering using YARN registry for MR AppMaster in YARN-3972 and want to take some learnings from here.
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| -1 | patch | 0m 0s | The patch command could not apply the patch during dryrun. |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12697782/YARN-2571-010.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / eee0d45 |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8905/console |
This message was automatically generated.
All the registry code is there (slider uses it), except two bits of the RM side
- create the base user path on app launch (in case the app needs it). This needs to be done by a process with the right permissions on ZK; it also makes sure that the user path is created with the perms to allow the RM/admin to delete it
- purge entries on container/AM failure
There was push-back from the YARN team on #2; not for the RM. I do still think #2 is needed. Irrespective of that, there is a main() entry point in the 2.7+ code which offers a CLI to create the reg; its just without docs or tests. Email me direct if you want to start using the code & I'll help you
Thanks Steve Loughran. I will ping you directly.
GitHub user steveloughran opened a pull request:
https://github.com/apache/hadoop/pull/66
YARN-2571 RM to support YARN registry
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveloughran/hadoop YARN-913/YARN-2571-RM-on-trunk
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hadoop/pull/66.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #66
commit 25a56da6037bb50d3ce5bbcc3001914e51ea2457
Author: Steve Loughran <stevel@hortonworks.com>
Date: 2015-11-11T20:20:23Z
YARN-2571 RM setup of registry, reapplied to trunk
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 0s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 13 new or modified test files. |
| +1 | mvninstall | 8m 18s | trunk passed |
| +1 | compile | 2m 10s | trunk passed with JDK v1.8.0_66 |
| +1 | compile | 2m 22s | trunk passed with JDK v1.7.0_91 |
| +1 | checkstyle | 0m 31s | trunk passed |
| +1 | mvnsite | 1m 27s | trunk passed |
| +1 | mvneclipse | 0m 44s | trunk passed |
| +1 | findbugs | 3m 6s | trunk passed |
| -1 | javadoc | 0m 33s | hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. |
| +1 | javadoc | 1m 25s | trunk passed with JDK v1.7.0_91 |
| -1 | mvninstall | 0m 36s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | compile | 2m 16s | hadoop-yarn in the patch failed with JDK v1.8.0_66. |
| -1 | javac | 2m 16s | hadoop-yarn in the patch failed with JDK v1.8.0_66. |
| -1 | compile | 2m 12s | hadoop-yarn in the patch failed with JDK v1.7.0_91. |
| -1 | javac | 2m 12s | hadoop-yarn in the patch failed with JDK v1.7.0_91. |
| -1 | checkstyle | 0m 37s | Patch generated 50 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 140, now 184). |
| -1 | mvnsite | 0m 38s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | mvneclipse | 0m 47s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | xml | 0m 0s | The patch has no ill-formed XML file. |
| -1 | findbugs | 0m 37s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | javadoc | 0m 31s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. |
| -1 | javadoc | 0m 15s | hadoop-yarn-registry in the patch failed with JDK v1.8.0_66. |
| +1 | javadoc | 1m 15s | the patch passed with JDK v1.7.0_91 |
| +1 | unit | 0m 29s | hadoop-yarn-server-common in the patch passed with JDK v1.8.0_66. |
| -1 | unit | 0m 36s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. |
| +1 | unit | 1m 15s | hadoop-yarn-registry in the patch passed with JDK v1.8.0_66. |
| +1 | unit | 0m 32s | hadoop-yarn-server-common in the patch passed with JDK v1.7.0_91. |
| -1 | unit | 0m 35s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. |
| +1 | unit | 1m 18s | hadoop-yarn-registry in the patch passed with JDK v1.7.0_91. |
| +1 | asflicense | 0m 23s | Patch does not generate ASF License warnings. |
| 41m 55s |
This message was automatically generated.
Patch 012: rebase to trunk
This patch is coming up to second birthday. Can someone pleae look at at it. Thanks
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 21s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 13 new or modified test files. |
| 0 | mvndep | 0m 48s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 51s | trunk passed |
| +1 | compile | 2m 17s | trunk passed |
| +1 | checkstyle | 0m 46s | trunk passed |
| +1 | mvnsite | 1m 18s | trunk passed |
| +1 | mvneclipse | 0m 43s | trunk passed |
| +1 | findbugs | 1m 56s | trunk passed |
| +1 | javadoc | 0m 50s | trunk passed |
| 0 | mvndep | 0m 9s | Maven dependency ordering for patch |
| -1 | mvninstall | 0m 28s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | compile | 1m 39s | hadoop-yarn in the patch failed. |
| -1 | javac | 1m 39s | hadoop-yarn in the patch failed. |
| -1 | checkstyle | 0m 46s | hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 696 unchanged - 10 fixed = 748 total (was 706) |
| -1 | mvnsite | 0m 29s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | mvneclipse | 0m 35s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | xml | 0m 2s | The patch has no ill-formed XML file. |
| -1 | findbugs | 0m 23s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | javadoc | 0m 11s | hadoop-yarn-registry in the patch failed. |
| +1 | unit | 0m 49s | hadoop-yarn-registry in the patch passed. |
| +1 | unit | 0m 25s | hadoop-yarn-server-common in the patch passed. |
| -1 | unit | 0m 28s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 26m 17s |
This message was automatically generated.
Patch 013
- fix compile problem against new yarn test
- do as much as possible to shut up checkstyle and javadoc
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 17s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 13 new or modified test files. |
| 0 | mvndep | 0m 9s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 39s | trunk passed |
| +1 | compile | 2m 17s | trunk passed |
| +1 | checkstyle | 0m 46s | trunk passed |
| +1 | mvnsite | 1m 26s | trunk passed |
| +1 | mvneclipse | 0m 47s | trunk passed |
| +1 | findbugs | 2m 19s | trunk passed |
| +1 | javadoc | 0m 53s | trunk passed |
| 0 | mvndep | 0m 11s | Maven dependency ordering for patch |
| -1 | mvninstall | 0m 34s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | compile | 1m 59s | hadoop-yarn in the patch failed. |
| -1 | javac | 1m 59s | hadoop-yarn in the patch failed. |
| -1 | checkstyle | 0m 49s | hadoop-yarn-project/hadoop-yarn: The patch generated 52 new + 695 unchanged - 10 fixed = 747 total (was 705) |
| -1 | mvnsite | 0m 35s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | mvneclipse | 0m 39s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | xml | 0m 2s | The patch has no ill-formed XML file. |
| -1 | findbugs | 0m 27s | hadoop-yarn-server-resourcemanager in the patch failed. |
| -1 | javadoc | 0m 12s | hadoop-yarn-registry in the patch failed. |
| +1 | unit | 0m 47s | hadoop-yarn-registry in the patch passed. |
| +1 | unit | 0m 26s | hadoop-yarn-server-common in the patch passed. |
| -1 | unit | 0m 29s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 27m 13s |
This message was automatically generated.
YARN-2517 patch 015: remove a test which was obsolete, but which didn't show as it was broken. Fix up more javadocs
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 20s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 14 new or modified test files. |
| 0 | mvndep | 0m 9s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 49s | trunk passed |
| +1 | compile | 2m 20s | trunk passed |
| +1 | checkstyle | 0m 50s | trunk passed |
| +1 | mvnsite | 1m 41s | trunk passed |
| +1 | mvneclipse | 0m 53s | trunk passed |
| 0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| +1 | findbugs | 2m 18s | trunk passed |
| +1 | javadoc | 1m 4s | trunk passed |
| 0 | mvndep | 0m 10s | Maven dependency ordering for patch |
| +1 | mvninstall | 1m 19s | the patch passed |
| +1 | compile | 2m 47s | the patch passed |
| -1 | javac | 2m 47s | hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35) |
| -1 | checkstyle | 0m 49s | hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 788 unchanged - 168 fixed = 803 total (was 956) |
| +1 | mvnsite | 1m 40s | the patch passed |
| +1 | mvneclipse | 0m 51s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| -1 | whitespace | 0m 0s | The patch 4 line(s) with tabs. |
| +1 | xml | 0m 1s | The patch has no ill-formed XML file. |
| 0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| +1 | findbugs | 2m 32s | the patch passed |
| +1 | javadoc | 0m 11s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52) |
| +1 | javadoc | 0m 13s | hadoop-yarn-server-common in the patch passed. |
| +1 | javadoc | 0m 19s | hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 | javadoc | 0m 8s | hadoop-yarn-site in the patch passed. |
| +1 | unit | 0m 48s | hadoop-yarn-registry in the patch passed. |
| +1 | unit | 0m 26s | hadoop-yarn-server-common in the patch passed. |
| +1 | unit | 37m 36s | hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 | unit | 0m 7s | hadoop-yarn-site in the patch passed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 67m 47s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | YARN-2571 |
| GITHUB PR | https://github.com/apache/hadoop/pull/66 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml |
| uname | Linux 8d6e2cd8e6db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 9faccd1 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| javac | https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn.txt |
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt |
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-eol.txt |
| whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/13086/artifact/patchprocess/whitespace-tabs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/13086/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U: hadoop-yarn-project/hadoop-yarn |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/13086/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
patch 016, fixup of final javadoc warnings, cleanup of <String> style definitions, and make sure public/private/evolving scope is everywhere appropriate, including package infos
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 20s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 14 new or modified test files. |
| 0 | mvndep | 0m 9s | Maven dependency ordering for branch |
| +1 | mvninstall | 6m 40s | trunk passed |
| +1 | compile | 2m 16s | trunk passed |
| +1 | checkstyle | 0m 50s | trunk passed |
| +1 | mvnsite | 1m 30s | trunk passed |
| +1 | mvneclipse | 0m 51s | trunk passed |
| 0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| +1 | findbugs | 1m 53s | trunk passed |
| +1 | javadoc | 0m 57s | trunk passed |
| 0 | mvndep | 0m 9s | Maven dependency ordering for patch |
| +1 | mvninstall | 1m 8s | the patch passed |
| +1 | compile | 2m 13s | the patch passed |
| -1 | javac | 2m 13s | hadoop-yarn-project_hadoop-yarn generated 1 new + 34 unchanged - 1 fixed = 35 total (was 35) |
| -1 | checkstyle | 0m 47s | hadoop-yarn-project/hadoop-yarn: The patch generated 15 new + 787 unchanged - 168 fixed = 802 total (was 955) |
| +1 | mvnsite | 1m 22s | the patch passed |
| +1 | mvneclipse | 0m 44s | the patch passed |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| -1 | whitespace | 0m 0s | The patch 4 line(s) with tabs. |
| +1 | xml | 0m 1s | The patch has no ill-formed XML file. |
| 0 | findbugs | 0m 0s | Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| +1 | findbugs | 2m 11s | the patch passed |
| +1 | javadoc | 0m 11s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-registry generated 0 new + 17 unchanged - 35 fixed = 17 total (was 52) |
| +1 | javadoc | 0m 13s | hadoop-yarn-server-common in the patch passed. |
| +1 | javadoc | 0m 19s | hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 | javadoc | 0m 7s | hadoop-yarn-site in the patch passed. |
| +1 | unit | 0m 41s | hadoop-yarn-registry in the patch passed. |
| +1 | unit | 0m 26s | hadoop-yarn-server-common in the patch passed. |
| -1 | unit | 33m 37s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | unit | 0m 7s | hadoop-yarn-site in the patch passed. |
| +1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| 61m 3s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
This message was automatically generated.
Patch 017, sync up with trunk
Can I observe that this patch is approaching its third birthday. I would really like to see it in, so that you can use the registry in a secure cluster. I'm happy to accept feedback and rework it, but we need to get this in.
I need reviewers here. Junping Du, are you able to look at it?
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 0s | Docker mode activated. |
| -1 | patch | 0m 5s | YARN-2571 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. |
| Subsystem | Report/Notes |
|---|---|
| JIRA Issue | YARN-2571 |
| GITHUB PR | https://github.com/apache/hadoop/pull/66 |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/16381/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Sure. Steve Loughran. I can help here.
It sounds like the patch cannot apply to trunk again, can you rebase to current trunk? Thx!
This is the patch of everything under hadoop-yarn-server to integrate the registry with the RM. The RM takes on the tasks of