Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 2.8.0, 2.6.5, 2.7.4, 3.0.0-alpha1
-
Component/s: resourcemanager
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
It is observed that RM triggers an one event for every ApplicationMaster#allocate request in the following trace. This is not necessarily required and it can be optimized such that send only if any containers are there to acknowledge to NodeManager.
RMAppAttemptImpl.sendFinishedContainersToNM() line: 1871 RMAppAttemptImpl.pullJustFinishedContainers() line: 805 ApplicationMasterService.allocate(AllocateRequest) line: 567
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 25s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 20s | trunk passed |
| +1 | compile | 0m 30s | trunk passed |
| +1 | checkstyle | 0m 22s | trunk passed |
| +1 | mvnsite | 0m 33s | trunk passed |
| +1 | mvneclipse | 0m 12s | trunk passed |
| +1 | findbugs | 0m 52s | trunk passed |
| +1 | javadoc | 0m 20s | trunk passed |
| +1 | mvninstall | 0m 28s | the patch passed |
| +1 | compile | 0m 26s | the patch passed |
| +1 | javac | 0m 26s | the patch passed |
| +1 | checkstyle | 0m 19s | the patch passed |
| +1 | mvnsite | 0m 30s | the patch passed |
| +1 | mvneclipse | 0m 10s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 | findbugs | 1m 0s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 | javadoc | 0m 16s | the patch passed |
| +1 | unit | 35m 23s | hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| 49m 0s |
| Reason | Tests |
|---|---|
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Nullcheck of finishedContainers at line 818 of value previously dereferenced in org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.pullJustFinishedContainers() At RMAppAttemptImpl.java:818 of value previously dereferenced in org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.pullJustFinishedContainers() At RMAppAttemptImpl.java:[line 818] |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:e2f6409 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12811145/0001-YARN-5262.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux b60589ea9cf0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / c9e7138 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/12038/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12038/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12038/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
Updated patch with fixing findbug warning
 +1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | reexec | 0m 26s | Docker mode activated. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | mvninstall | 6m 33s | trunk passed |
| +1 | compile | 0m 33s | trunk passed |
| +1 | checkstyle | 0m 22s | trunk passed |
| +1 | mvnsite | 0m 35s | trunk passed |
| +1 | mvneclipse | 0m 14s | trunk passed |
| +1 | findbugs | 0m 57s | trunk passed |
| +1 | javadoc | 0m 21s | trunk passed |
| +1 | mvninstall | 0m 29s | the patch passed |
| +1 | compile | 0m 28s | the patch passed |
| +1 | javac | 0m 28s | the patch passed |
| +1 | checkstyle | 0m 19s | the patch passed |
| +1 | mvnsite | 0m 33s | the patch passed |
| +1 | mvneclipse | 0m 12s | the patch passed |
| +1 | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 | findbugs | 1m 2s | the patch passed |
| +1 | javadoc | 0m 17s | the patch passed |
| +1 | unit | 35m 20s | hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 | asflicense | 0m 15s | The patch does not generate ASF License warnings. |
| 49m 34s |
| Subsystem | Report/Notes |
|---|---|
| Docker | Image:yetus/hadoop:e2f6409 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12811277/0002-YARN-5262.patch |
| JIRA Issue | |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux b6119aa58446 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 51d497f |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/12058/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/12058/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
committed to trunk/branch-2!! thanks Jian He for the review
SUCCESS: Integrated in Hadoop-trunk-Commit #10031 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10031/)
YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for (rohithsharmaks: rev 26b5e6116f392b6be91dca57968259f87554ce33)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
Thanks, Rohith Sharma K S! I committed this to branch-2.8, branch-2.7, and branch-2.6 as well.
Updated the straight forward patch..