Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.8.0
-
Fix Version/s: 2.9.0, 3.0.0-alpha1
-
Component/s: fairscheduler
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
The maxAMShare uses the queue's policy for enforcing limits. When using FAIR policy, this considers only memory. If there are fewer vcores on the cluster, the AMs can end up taking all the vcores leading to a livelock.
Issue Links
- duplicates
-
YARN-3621
FairScheduler doesn't count AM vcores towards max-share
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 13s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 8m 29s | trunk passed |
| +1 | compile | 0m 37s | trunk passed with JDK v1.8.0_91 |
| +1 | compile | 0m 36s | trunk passed with JDK v1.7.0_95 |
| +1 | checkstyle | 0m 31s | trunk passed |
| +1 | mvnsite | 0m 43s | trunk passed |
| +1 | mvneclipse | 0m 17s | trunk passed |
| +1 | findbugs | 1m 20s | trunk passed |
| +1 | javadoc | 0m 30s | trunk passed with JDK v1.8.0_91 |
| +1 | javadoc | 0m 32s | trunk passed with JDK v1.7.0_95 |
| +1 | mvninstall | 0m 37s | the patch passed |
| +1 | compile | 0m 35s | the patch passed with JDK v1.8.0_91 |
| +1 | javac | 0m 35s | the patch passed |
| +1 | compile | 0m 33s | the patch passed with JDK v1.7.0_95 |
| +1 | javac | 0m 33s | the patch passed |
| +1 | checkstyle | 0m 28s | the patch passed |
| +1 | mvnsite | 0m 41s | the patch passed |
| +1 | mvneclipse | 0m 15s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| -1 | findbugs | 1m 38s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 | javadoc | 0m 30s | the patch passed with JDK v1.8.0_91 |
| +1 | javadoc | 0m 30s | the patch passed with JDK v1.7.0_95 |
| -1 | unit | 35m 14s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. |
| -1 | unit | 35m 17s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. |
| +1 | asflicense | 0m 21s | Patch does not generate ASF License warnings. |
| 91m 37s |
| Reason | Tests |
|---|---|
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.rootMetrics; locked 80% of time Unsynchronized access at FairScheduler.java:80% of time Unsynchronized access at FairScheduler.java:[line 1202] | |
| JDK v1.8.0_91 Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | |
| hadoop.yarn.server.resourcemanager.TestContainerResourceUsage | |
| JDK v1.7.0_95 Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestContinuousScheduling | |
| hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | |
| hadoop.yarn.server.resourcemanager.TestContainerResourceUsage |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 17s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 8m 53s | trunk passed |
| +1 | compile | 0m 39s | trunk passed with JDK v1.8.0_91 |
| +1 | compile | 0m 39s | trunk passed with JDK v1.7.0_95 |
| +1 | checkstyle | 0m 33s | trunk passed |
| +1 | mvnsite | 0m 45s | trunk passed |
| +1 | mvneclipse | 0m 19s | trunk passed |
| +1 | findbugs | 1m 29s | trunk passed |
| +1 | javadoc | 0m 36s | trunk passed with JDK v1.8.0_91 |
| +1 | javadoc | 0m 36s | trunk passed with JDK v1.7.0_95 |
| +1 | mvninstall | 0m 41s | the patch passed |
| +1 | compile | 0m 39s | the patch passed with JDK v1.8.0_91 |
| +1 | javac | 0m 39s | the patch passed |
| +1 | compile | 0m 37s | the patch passed with JDK v1.7.0_95 |
| +1 | javac | 0m 37s | the patch passed |
| -1 | checkstyle | 0m 29s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 212 unchanged - 1 fixed = 213 total (was 213) |
| +1 | mvnsite | 0m 41s | the patch passed |
| +1 | mvneclipse | 0m 17s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| +1 | findbugs | 1m 41s | the patch passed |
| +1 | javadoc | 0m 31s | the patch passed with JDK v1.8.0_91 |
| +1 | javadoc | 0m 33s | the patch passed with JDK v1.7.0_95 |
| -1 | unit | 37m 33s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. |
| -1 | unit | 36m 9s | hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. |
| +1 | asflicense | 0m 22s | Patch does not generate ASF License warnings. |
| 96m 14s |
| Reason | Tests |
|---|---|
| JDK v1.8.0_91 Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority |
| hadoop.yarn.server.resourcemanager.TestClientRMTokens | |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | |
| hadoop.yarn.server.resourcemanager.TestContainerResourceUsage | |
| JDK v1.7.0_95 Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer | |
| hadoop.yarn.server.resourcemanager.TestContainerResourceUsage |
This message was automatically generated.
Test failures are unrelated to the patch.
Thanks for working on this, Yufei. Comments on the patch:
- In addition to checking the cluster resources, should we also check against queue's max resources for this queue and any parents? For the latter, a helper method might be needed.
- Nit: Can we avoid naming the variables with a "not" for readability? For instance, may be we could use overMaxAMShareLimit instead of notOverLimit and exhaustsVcores instead of notTakeAllVCore and negate the expressions as necessary?
- Nit: Also, do we really need the second boolean. How about modifying the newly added if condition to if (!overMaxAMShareLimit && policy is not DRF) and just override overMaxAMShareLimit? The method itself could return !overMaxAMShareLimit?
- Nit: After the changes, if the code needs more explanation, should we add a comment before the if condition so it is clear why the second check?
Thanks Karthik Kambatla for reviewing.
I upload a new patch for all suggestions.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 12s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 8m 59s | trunk passed |
| +1 | compile | 0m 35s | trunk passed |
| +1 | checkstyle | 0m 28s | trunk passed |
| +1 | mvnsite | 0m 45s | trunk passed |
| +1 | mvneclipse | 0m 15s | trunk passed |
| +1 | findbugs | 1m 12s | trunk passed |
| +1 | javadoc | 0m 28s | trunk passed |
| +1 | mvninstall | 0m 37s | the patch passed |
| +1 | compile | 0m 34s | the patch passed |
| +1 | javac | 0m 34s | the patch passed |
| -1 | checkstyle | 0m 23s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 213 unchanged - 1 fixed = 214 total (was 214) |
| +1 | mvnsite | 0m 41s | the patch passed |
| +1 | mvneclipse | 0m 14s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| -1 | findbugs | 1m 23s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 | javadoc | 0m 24s | the patch passed |
| -1 | unit | 31m 26s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 19s | Patch does not generate ASF License warnings. |
| 49m 38s |
| Reason | Tests |
|---|---|
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Null pointer dereference of queue in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) Dereferenced at FSLeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) Dereferenced at FSLeafQueue.java:[line 523] | |
| Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) At FSLeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) At FSLeafQueue.java:[line 522] | |
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.TestClientRMTokens |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 12s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 32s | trunk passed |
| +1 | compile | 0m 32s | trunk passed |
| +1 | checkstyle | 0m 28s | trunk passed |
| +1 | mvnsite | 0m 43s | trunk passed |
| +1 | mvneclipse | 0m 15s | trunk passed |
| +1 | findbugs | 1m 9s | trunk passed |
| +1 | javadoc | 0m 25s | trunk passed |
| +1 | mvninstall | 0m 41s | the patch passed |
| +1 | compile | 0m 37s | the patch passed |
| +1 | javac | 0m 37s | the patch passed |
| -1 | checkstyle | 0m 26s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 213 unchanged - 1 fixed = 214 total (was 214) |
| +1 | mvnsite | 0m 41s | the patch passed |
| +1 | mvneclipse | 0m 13s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| -1 | findbugs | 1m 22s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 | javadoc | 0m 25s | the patch passed |
| -1 | unit | 31m 25s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 15s | Patch does not generate ASF License warnings. |
| 47m 3s |
| Reason | Tests |
|---|---|
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Null pointer dereference of queue in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) Dereferenced at FSLeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) Dereferenced at FSLeafQueue.java:[line 523] | |
| Load of known null value in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) At FSLeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.isVCoreOverMaxResource(Resource, FSQueue) At FSLeafQueue.java:[line 522] | |
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.TestClientRMTokens |
This message was automatically generated.
The patch 006 fixed the style issues and the bug, add some test code to test if AM VCores are more than maxResources of queue.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 11s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 21s | trunk passed |
| +1 | compile | 0m 30s | trunk passed |
| +1 | checkstyle | 0m 21s | trunk passed |
| +1 | mvnsite | 0m 33s | trunk passed |
| +1 | mvneclipse | 0m 12s | trunk passed |
| +1 | findbugs | 0m 56s | trunk passed |
| +1 | javadoc | 0m 22s | trunk passed |
| +1 | mvninstall | 0m 29s | the patch passed |
| +1 | compile | 0m 26s | the patch passed |
| +1 | javac | 0m 26s | the patch passed |
| +1 | checkstyle | 0m 19s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 212 unchanged - 1 fixed = 212 total (was 213) |
| +1 | mvnsite | 0m 32s | the patch passed |
| +1 | mvneclipse | 0m 10s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| +1 | findbugs | 0m 57s | the patch passed |
| +1 | javadoc | 0m 19s | the patch passed |
| -1 | unit | 29m 31s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 17s | Patch does not generate ASF License warnings. |
| 42m 59s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12805079/YARN-4866.006.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux c5418cd73084 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 42c22f7 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11574/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/11574/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11574/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11574/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
All test failures are unrelated.
Patch looks close.
- Shouldn't isVCoreOverMaxResource be added to FSQueue and not FSLeafQueue as a protected method? That way, we don't have to pass the queue.
- Nit: Javadoc for FSLeafQueue#canRunAppAM mentions Fifo and Fair policy don't care about vcores. I would probably reword and reuse the comment closer to the check - " Check whether this queue can run this application master under the maxAMShare limit. For FIFO and FAIR policies, check if the vcore usage takes up the entire cluster or maxResources for the queue".
- Would the check for FAIR and FIFO in the method be more readable if we change the order of boolean expressions?
overMaxAMShareLimit = isVCoreOverMaxResource(ifRunAMResource) || ifRunAMResource.getVirtualCores() >= scheduler.getRootQueueMetrics().getAvailableVirtualCores();
Thanks Karthik Kambatla for the detailed review.
1. The parameter queue is used for recursive checking. It will be there if we put this function in FSQueue. In that case, no obvious reason to move the function to FSQueue, unless FSParentQueue need it in the future.
2. 3. Nice suggestions. Thanks. I've uploaded patch 007 for it.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 10s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 18s | trunk passed |
| +1 | compile | 0m 28s | trunk passed |
| +1 | checkstyle | 0m 23s | trunk passed |
| +1 | mvnsite | 0m 32s | trunk passed |
| +1 | mvneclipse | 0m 12s | trunk passed |
| +1 | findbugs | 0m 51s | trunk passed |
| +1 | javadoc | 0m 20s | trunk passed |
| +1 | mvninstall | 0m 28s | the patch passed |
| +1 | compile | 0m 30s | the patch passed |
| +1 | javac | 0m 30s | the patch passed |
| +1 | checkstyle | 0m 22s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 212 unchanged - 1 fixed = 212 total (was 213) |
| +1 | mvnsite | 0m 35s | the patch passed |
| +1 | mvneclipse | 0m 12s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| +1 | findbugs | 1m 10s | the patch passed |
| +1 | javadoc | 0m 22s | the patch passed |
| -1 | unit | 30m 11s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 17s | Patch does not generate ASF License warnings. |
| 43m 58s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestContainerResourceUsage |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12806031/YARN-4866.007.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 01a4005539dc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / edd716e |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11680/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/11680/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11680/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11680/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
The parameter queue is used for recursive checking. It will be there if we put this function in FSQueue. In that case, no obvious reason to move the function to FSQueue, unless FSParentQueue need it in the future.
Not sure I follow. Would the following not work when implemented in FSQueue?
boolean isVcoresOverMaxResource(Resource resource) { if (resource.getVirtualCores() > scheduler.getAllocationConfiguration() .getMaxResources(getName()).getVirtualCores()) { return true; } FSQueue parent = getParent(); if (parent == null) { return false; } return parent().isVcoresOverMaxResource(); }
Karthik Kambatla, Got you. Thanks for detailed explanation. Sorry for the misunderstanding. I uploaded patch 008 with the modification you suggested.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 13s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 9m 33s | trunk passed |
| +1 | compile | 0m 37s | trunk passed |
| +1 | checkstyle | 0m 28s | trunk passed |
| +1 | mvnsite | 0m 45s | trunk passed |
| +1 | mvneclipse | 0m 14s | trunk passed |
| +1 | findbugs | 1m 29s | trunk passed |
| +1 | javadoc | 0m 27s | trunk passed |
| +1 | mvninstall | 0m 38s | the patch passed |
| +1 | compile | 0m 37s | the patch passed |
| +1 | javac | 0m 37s | the patch passed |
| -1 | checkstyle | 0m 25s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 224 unchanged - 1 fixed = 225 total (was 225) |
| +1 | mvnsite | 0m 43s | the patch passed |
| +1 | mvneclipse | 0m 12s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| +1 | findbugs | 1m 40s | the patch passed |
| -1 | javadoc | 2m 24s | hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager generated 2 new + 1255 unchanged - 0 fixed = 1257 total (was 1255) |
| +1 | javadoc | 0m 27s | the patch passed |
| -1 | unit | 31m 31s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 16s | Patch does not generate ASF License warnings. |
| 51m 1s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| hadoop.yarn.server.resourcemanager.TestClientRMTokens |
This message was automatically generated.
Uploaded patch 009 to fix style and javadoc issues.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 14s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 46s | trunk passed |
| +1 | compile | 0m 29s | trunk passed |
| +1 | checkstyle | 0m 22s | trunk passed |
| +1 | mvnsite | 0m 32s | trunk passed |
| +1 | mvneclipse | 0m 12s | trunk passed |
| +1 | findbugs | 0m 53s | trunk passed |
| +1 | javadoc | 0m 22s | trunk passed |
| +1 | mvninstall | 0m 30s | the patch passed |
| +1 | compile | 0m 28s | the patch passed |
| +1 | javac | 0m 28s | the patch passed |
| +1 | checkstyle | 0m 19s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 0 new + 223 unchanged - 1 fixed = 223 total (was 224) |
| +1 | mvnsite | 0m 32s | the patch passed |
| +1 | mvneclipse | 0m 10s | the patch passed |
| +1 | whitespace | 0m 0s | Patch has no whitespace issues. |
| +1 | findbugs | 0m 59s | the patch passed |
| +1 | javadoc | 0m 19s | the patch passed |
| -1 | unit | 34m 42s | hadoop-yarn-server-resourcemanager in the patch failed. |
| +1 | asflicense | 0m 15s | Patch does not generate ASF License warnings. |
| 48m 41s |
| Reason | Tests |
|---|---|
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMRestart |
| hadoop.yarn.server.resourcemanager.TestClientRMTokens | |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12806184/YARN-4866.009.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 95e0a79108ce 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 77d5ce9 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| unit | https://builds.apache.org/job/PreCommit-YARN-Build/11693/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-YARN-Build/11693/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/11693/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/11693/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
+1. Checking this in.
SUCCESS: Integrated in Hadoop-trunk-Commit #9864 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9864/)
YARN-4866. FairScheduler: AMs can consume all vcores leading to a (kasha: rev 4f513a4a8ed73beec89b7b28c0cd056ee5f4ed0d)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSQueue.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java
Just committed this to trunk and branch-2.
Thanks for the contribution, Yufei Gu.
Thanks a lot for the review and commit, Karthik Kambatla.
YARN-4866This message was automatically generated.