Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
Add node/host to attribute mapping in yarn client API.
Attachments
Attachments
- YARN-8104-YARN-3409.006.patch
- 59 kB
- Bibin Chundatt
- YARN-8104-YARN-3409.005.patch
- 59 kB
- Bibin Chundatt
- YARN-8104-YARN-3409.004.patch
- 59 kB
- Bibin Chundatt
- YARN-8104-YARN-3409.003.patch
- 56 kB
- Bibin Chundatt
- YARN-8104-YARN-3409.002.patch
- 56 kB
- Bibin Chundatt
- YARN-8104-YARN-3409.001.patch
- 49 kB
- Bibin Chundatt
Activity
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 0s | Docker mode activated. |
-1 | patch | 0m 6s | |
Subsystem | Report/Notes |
---|---|
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12917543/YARN-8104-YARN-3409.001.patch |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/20218/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Hi bibinchundatt, Now that YARN-8100 is in can you rebase on this ?
Also there was one point i thought of adding in 8100 , GetAttributesToNodesResponseProto could have had field attributeToNodes as attributesToNodes.
Can you incorporate that here ?
Thank you Naganarasimha for review
Attached rebased patch and also handling your comments.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 37s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
|
|||
0 | mvndep | 1m 25s | Maven dependency ordering for branch |
+1 | mvninstall | 26m 23s | |
+1 | compile | 30m 3s | |
+1 | checkstyle | 3m 15s | |
+1 | mvnsite | 5m 0s | |
+1 | shadedclient | 19m 15s | branch has no errors when building and testing our client artifacts. |
-1 | findbugs | 1m 20s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in |
+1 | javadoc | 3m 50s | |
Patch Compile Tests | |||
0 | mvndep | 0m 19s | Maven dependency ordering for patch |
+1 | mvninstall | 3m 54s | the patch passed |
+1 | compile | 29m 34s | the patch passed |
+1 | cc | 29m 34s | the patch passed |
+1 | javac | 29m 34s | the patch passed |
-0 | checkstyle | 3m 25s | root: The patch generated 6 new + 261 unchanged - 0 fixed = 267 total (was 261) |
+1 | mvnsite | 4m 56s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 10m 28s | patch has no errors when building and testing our client artifacts. |
-1 | findbugs | 1m 34s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
-1 | javadoc | 0m 32s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) |
-1 | javadoc | 0m 56s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common generated 1 new + 4189 unchanged - 0 fixed = 4190 total (was 4189) |
Other Tests | |||
+1 | unit | 0m 50s | hadoop-yarn-api in the patch passed. |
+1 | unit | 3m 14s | hadoop-yarn-common in the patch passed. |
+1 | unit | 2m 14s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 66m 57s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | unit | 27m 3s | hadoop-yarn-client in the patch passed. |
+1 | unit | 1m 29s | hadoop-yarn-server-router in the patch passed. |
-1 | unit | 127m 42s | hadoop-mapreduce-client-jobclient in the patch failed. |
+1 | asflicense | 0m 48s | The patch does not generate ASF License warnings. |
387m 23s |
Reason | Tests |
---|---|
FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
Nullcheck of hostName at line 454 of value previously dereferenced in org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesManagerImpl.getNodesToAttributes(Set) At NodeAttributesManagerImpl.java:454 of value previously dereferenced in org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesManagerImpl.getNodesToAttributes(Set) At NodeAttributesManagerImpl.java:[line 454] |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12918329/YARN-8104-YARN-3409.003.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc |
uname | Linux 8a5fa31c02ec 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html |
checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/diff-checkstyle-root.txt |
findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html |
javadoc | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt |
javadoc | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/20285/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt |
Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/20285/testReport/ |
Max. process+thread count | 1211 (vs. ulimit of 10000) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/20285/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Thanks for the patch bibinchundatt
At high level
- Can you inform why NodeToAttributesProto is moved from yarn_server_resourcemanager_service_protos.proto to yarn_protos.proto ?
- Also would it make sense to provide overloaded method(getNodesToAttributes) here supporting for NodeID ?
and few other comments :
yarn_protos.protos
- ln no 391: node => hostname ... similar to earlier naming convention
yarn_service_protos.protos
- ln no 282: nodeToAttributes => nodesToAttributes ... based on convention followed in other places
GetNodesToAttributesRequest.java
- line no 55 & 64 : setNodes & getNodes -> setHostNames & getHostNames
GetNodesToAttributesRequestPBImpl
- line no 120: initNodeAttributes => initNodesToAttributesRequest or just init
- line no 126: nodeLabelsList => hostNamesList
TestPBImplRecords
- We need to invoke generateByNewInstance for all the new PB's in setup. can you please check.
NodeAttributesManagerImpl
- ln no 454-457: Here we do not have a mapping we are setting a hostname with empty set, is that better or just pass for the ones which have attributes is better?
IMO for the ones having mapping is better so that we not bloating the response and its a map. Also if nonexistent of erroneous hostnames are given it still shows empty set
TestClientRMService
- ln no 2053: can we have a separate method to test node to attributes API ? or document what all api's will be tested. and rename it to testNodeAttributesQueryAPI... i would still prefer the former option itself though..
Can you also check the new findbug issue reported, checktyle and as well as the javadoc issues reported, as it seems related to patch and fixable ?
Thank you Naganarasimha for review
Can you inform why NodeToAttributesProto is moved from yarn_server_resourcemanager_service_protos.proto to yarn_protos.proto
GetNodesToAttributesResponseProto yarn_service.proto requires the same yarn_protos.
Also would it make sense to provide overloaded method(getNodesToAttributes) here supporting for NodeID ?
Currently IMHO don't make sense since NodeAttributeManagerImpl doesn't use NodeId. We can take up this point once we support in NodeAttributeManager.
We need to invoke generateByNewInstance for all the new PB's in setup. can you please check.
PB's are added as part of testGetNodesToAttributesRequestPBImpl and testGetNodesToAttributesResponsePBImpl
GetNodesToAttributesRequestPBImpl
Handled.
NodeAttributesManagerImpl
Here we do not have a mapping we are setting a hostname with empty set, is that better or just pass for the ones which have attributes is better?
Avoiding for which attribute mapping are not set.
TestClientRMService
Separated testcase for each.
Handled javadoc and checkstyle issues too.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 40s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
|
|||
0 | mvndep | 1m 21s | Maven dependency ordering for branch |
+1 | mvninstall | 25m 19s | |
+1 | compile | 29m 20s | |
+1 | checkstyle | 3m 14s | |
+1 | mvnsite | 5m 2s | |
+1 | shadedclient | 19m 14s | branch has no errors when building and testing our client artifacts. |
-1 | findbugs | 1m 23s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in |
+1 | javadoc | 3m 44s | |
Patch Compile Tests | |||
0 | mvndep | 0m 19s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 2s | the patch passed |
+1 | compile | 38m 55s | the patch passed |
+1 | cc | 38m 55s | the patch passed |
+1 | javac | 38m 55s | the patch passed |
-0 | checkstyle | 3m 43s | root: The patch generated 5 new + 261 unchanged - 0 fixed = 266 total (was 261) |
+1 | mvnsite | 6m 19s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 12m 5s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 9m 48s | the patch passed |
+1 | javadoc | 5m 0s | the patch passed |
Other Tests | |||
+1 | unit | 0m 56s | hadoop-yarn-api in the patch passed. |
+1 | unit | 3m 32s | hadoop-yarn-common in the patch passed. |
+1 | unit | 2m 33s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 71m 26s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | unit | 26m 58s | hadoop-yarn-client in the patch passed. |
+1 | unit | 1m 29s | hadoop-yarn-server-router in the patch passed. |
-1 | unit | 127m 10s | hadoop-mapreduce-client-jobclient in the patch failed. |
+1 | asflicense | 0m 46s | The patch does not generate ASF License warnings. |
405m 34s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12918512/YARN-8104-YARN-3409.004.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc |
uname | Linux 29a625d59cb3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/20298/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html |
checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/20298/artifact/out/diff-checkstyle-root.txt |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/20298/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt |
Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/20298/testReport/ |
Max. process+thread count | 1195 (vs. ulimit of 10000) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/20298/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 41s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
|
|||
0 | mvndep | 1m 29s | Maven dependency ordering for branch |
+1 | mvninstall | 27m 4s | |
+1 | compile | 28m 49s | |
+1 | checkstyle | 3m 14s | |
+1 | mvnsite | 4m 57s | |
+1 | shadedclient | 19m 7s | branch has no errors when building and testing our client artifacts. |
-1 | findbugs | 1m 19s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in |
+1 | javadoc | 3m 45s | |
Patch Compile Tests | |||
0 | mvndep | 0m 18s | Maven dependency ordering for patch |
+1 | mvninstall | 3m 41s | the patch passed |
+1 | compile | 43m 32s | the patch passed |
+1 | cc | 43m 32s | the patch passed |
+1 | javac | 43m 32s | the patch passed |
+1 | checkstyle | 6m 7s | the patch passed |
+1 | mvnsite | 9m 49s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 10m 31s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 7m 53s | the patch passed |
+1 | javadoc | 3m 46s | the patch passed |
Other Tests | |||
+1 | unit | 0m 47s | hadoop-yarn-api in the patch passed. |
+1 | unit | 3m 11s | hadoop-yarn-common in the patch passed. |
+1 | unit | 2m 12s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 66m 55s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | unit | 27m 1s | hadoop-yarn-client in the patch passed. |
+1 | unit | 1m 29s | hadoop-yarn-server-router in the patch passed. |
-1 | unit | 124m 56s | hadoop-mapreduce-client-jobclient in the patch failed. |
+1 | asflicense | 0m 51s | The patch does not generate ASF License warnings. |
404m 8s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12918587/YARN-8104-YARN-3409.005.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc |
uname | Linux 61ebc4154df9 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_151 |
findbugs | v3.1.0-RC1 |
findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/20305/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/20305/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt |
Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/20305/testReport/ |
Max. process+thread count | 1205 (vs. ulimit of 10000) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/20305/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Thanks for the patches bibinchundatt,
Currently IMHO don't make sense since NodeAttributeManagerImpl doesn't use NodeId. We can take up this point once we support in NodeAttributeManager.
I never meant the comment should change the API in NodeAttributeManager neither are there any plans in that direction. Till now the apis exposed are either internal or admin related. But for user whether it would make sense to have API for NodeId is the question here.
Also was wondering what would be the behavior if ip's are used instead of hostnames while mapping?
GetNodesToAttributesResponseProto yarn_service.proto requires the same yarn_protos.
agree
hadoop-mapreduce-client-jobclient unit tests either failed or timed out can you have a local run and share the results ?
Apart from that patch LGTM !
Got confused with NodeAttributesManagerImpl#getNodeToAttributes. IIUC you are focusing on YarnClient#getNodeToAttributes.
It does make sense from user's perspective since all other api to him are NodeID. But if user adds two nodemanagers in same machine.
(How valid is the deployment scenarios i am also not sure.) this API might create additional confusion rt ??
In distributed mode one Nodemanager can change attributes of second NM on same node?
IMHO need further discussion on this point. Thoughts on having follow up jira after discussion for Node ID related clarifications ?.
Regarding test timeout
org.apache.hadoop.mapred.pipes.TestPipeApplication is getting timeout in trunk too .
IMHO need further discussion on this point. Thoughts on having follow up jira after discussion for Node ID related clarifications ?.
It was discussed in detail and then only we came down to this decision, in none of enterprise clusters we have two NM's in the same node so it's only a testing scenario. Hence we have knowingly removed it. So i do not see any need to handle for a test scenario as of now unless we have good scenario we need not address it.
Thank you Naganarasimha for sharing.. Hope no further updates are required.
Yes mostly fine with me but would like sunil.govind@gmail.com to review once before i commit. Also for our offline discussion on hostname, IIUC its not necessary that nodes needs to be configured with hostnames, ip's can also be configured so in that sense would the api be good with hostname as the variable ? sunil.govind@gmail.com correct me if i am wrong.
IIUC ContainerManagerImpl#serviceStart registeration will happen always with hostname.
NodeId nodeId = buildNodeId(connectAddress, hostOverride); ((NodeManager.NMContext)context).setNodeId(nodeId); private NodeId buildNodeId(InetSocketAddress connectAddress, String hostOverride) { if (hostOverride != null) { connectAddress = NetUtils.getConnectAddress( new InetSocketAddress(hostOverride, connectAddress.getPort())); } return NodeId.newInstance( connectAddress.getAddress().getCanonicalHostName(), connectAddress.getPort()); }
As discussed offline updating patch with minor variable name change in proto
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 49s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 5 new or modified test files. |
|
|||
0 | mvndep | 2m 5s | Maven dependency ordering for branch |
+1 | mvninstall | 23m 53s | |
+1 | compile | 27m 14s | |
+1 | checkstyle | 2m 39s | |
+1 | mvnsite | 4m 26s | |
+1 | shadedclient | 16m 37s | branch has no errors when building and testing our client artifacts. |
-1 | findbugs | 1m 12s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in |
+1 | javadoc | 3m 38s | |
Patch Compile Tests | |||
0 | mvndep | 0m 18s | Maven dependency ordering for patch |
+1 | mvninstall | 3m 30s | the patch passed |
+1 | compile | 25m 37s | the patch passed |
+1 | cc | 25m 37s | the patch passed |
+1 | javac | 25m 37s | the patch passed |
+1 | checkstyle | 2m 43s | the patch passed |
+1 | mvnsite | 4m 47s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 9m 25s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 7m 40s | the patch passed |
+1 | javadoc | 3m 45s | the patch passed |
Other Tests | |||
+1 | unit | 0m 46s | hadoop-yarn-api in the patch passed. |
+1 | unit | 3m 20s | hadoop-yarn-common in the patch passed. |
+1 | unit | 2m 18s | hadoop-yarn-server-common in the patch passed. |
+1 | unit | 67m 30s | hadoop-yarn-server-resourcemanager in the patch passed. |
+1 | unit | 27m 18s | hadoop-yarn-client in the patch passed. |
+1 | unit | 1m 28s | hadoop-yarn-server-router in the patch passed. |
-1 | unit | 129m 21s | hadoop-mapreduce-client-jobclient in the patch failed. |
+1 | asflicense | 0m 35s | The patch does not generate ASF License warnings. |
373m 55s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.mapred.TestMRTimelineEventHandling |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12918969/YARN-8104-YARN-3409.006.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc |
uname | Linux 1973c71f1dd7 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31 19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_162 |
findbugs | v3.1.0-RC1 |
findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/20339/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-warnings.html |
unit | https://builds.apache.org/job/PreCommit-YARN-Build/20339/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt |
Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/20339/testReport/ |
Max. process+thread count | 1293 (vs. ulimit of 10000) |
modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . |
Console output | https://builds.apache.org/job/PreCommit-YARN-Build/20339/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14931 (See https://builds.apache.org/job/Hadoop-trunk-Commit/14931/)
YARN-8104. Add API to fetch node to attribute mapping. Contributed by (sunilg: rev 5dc7d6e0f3f46277f0615d61755970a7aa28b8d7)
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/RouterClientRMService.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto
- (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNodesToAttributesRequestPBImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestPBImplRecords.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeAttributesManager.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NodeAttributesManagerImpl.java
- (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNodesToAttributesRequest.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetAttributesToNodesResponsePBImpl.java
- (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetNodesToAttributesResponsePBImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodesToAttributesMappingRequestPBImpl.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/DefaultClientRequestInterceptor.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/MockResourceManagerFacade.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java
- (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetNodesToAttributesResponse.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/server/yarn_server_resourcemanager_service_protos.proto
- (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/PassThroughClientRequestInterceptor.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java
- (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/impl/pb/NodeToAttributesPBImpl.java
Attached first patch.
sunil.govind@gmail.com/Naganarasimha/cheersyang please help in review