Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 2.6.1, 2.8.0, 2.7.2, 3.0.0-alpha1
-
Component/s: None
-
Labels:
-
Target Version/s:
Description
If external systems like ATS, or ZK becomes very slow, draining all the events take a lot of time. If this time becomes larger than 10 mins, all applications will expire. Fixes include:
1. add a timeout and stop the dispatcher even if not all events are drained.
2. Move ATS service out from RM active service so that RM doesn't need to wait for ATS to flush the events when transitioning to standby.
3. Stop client-facing services (ClientRMService etc.) first so that clients get fast notification that RM is stopping/transitioning.
Issue Links
- is related to
-
YARN-4153
TestAsyncDispatcher failed at branch-2.7
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 17m 48s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | javac | 7m 56s | There were no new javac warning messages. |
| +1 | javadoc | 9m 44s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 48s | The applied patch generated 1 new checkstyle issues (total was 50, now 50). |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 22s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 3m 3s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | yarn tests | 2m 3s | Tests passed in hadoop-yarn-common. |
| -1 | yarn tests | 43m 41s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 88m 24s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestApplicationCleanup |
| hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification | |
| hadoop.yarn.server.resourcemanager.TestApplicationMasterService | |
| hadoop.yarn.server.resourcemanager.TestRMAdminService | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority | |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens | |
| hadoop.yarn.server.resourcemanager.TestClientRMService | |
| hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter | |
| hadoop.yarn.server.resourcemanager.TestRMHA | |
| hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart | |
| hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption | |
| hadoop.yarn.server.resourcemanager.security.TestAMRMTokens |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12748093/YARN-3999.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 88d8736 |
| checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/8726/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt |
| hadoop-yarn-common test log | https://builds.apache.org/job/PreCommit-YARN-Build/8726/artifact/patchprocess/testrun_hadoop-yarn-common.txt |
| hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8726/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8726/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8726/console |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| -1 | pre-patch | 15m 34s | Findbugs (version ) appears to be broken on trunk. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 1 new or modified test files. |
| +1 | javac | 7m 43s | There were no new javac warning messages. |
| +1 | javadoc | 9m 36s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. |
| +1 | checkstyle | 0m 54s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 23s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 3m 0s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | yarn tests | 1m 56s | Tests passed in hadoop-yarn-common. |
| -1 | yarn tests | 42m 51s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 83m 55s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestClientRMService |
| hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification | |
| hadoop.yarn.server.resourcemanager.security.TestAMRMTokens | |
| hadoop.yarn.server.resourcemanager.TestRMHA | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens | |
| hadoop.yarn.server.resourcemanager.TestApplicationCleanup | |
| hadoop.yarn.server.resourcemanager.TestApplicationMasterService | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption | |
| hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter | |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler | |
| hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart | |
| hadoop.yarn.server.resourcemanager.TestRMAdminService |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12748093/YARN-3999.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 88d8736 |
| hadoop-yarn-common test log | https://builds.apache.org/job/PreCommit-YARN-Build/8730/artifact/patchprocess/testrun_hadoop-yarn-common.txt |
| hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/8730/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt |
| Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/8730/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8730/console |
This message was automatically generated.
Can we move
transitionToStandby(false);
before
super.serviceStop();
in that case, when we shut dow the RM, we transit RM to standby first (stop all the active services), then stop all the alway-on services ?
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 19m 0s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 7 new or modified test files. |
| +1 | javac | 8m 6s | There were no new javac warning messages. |
| +1 | javadoc | 10m 26s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 50s | The applied patch generated 7 new checkstyle issues (total was 88, now 89). |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 25s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 3m 13s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | yarn tests | 2m 3s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 41m 8s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 88m 10s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.yarn.util.TestRackResolver |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations | |
| hadoop.yarn.server.resourcemanager.security.TestAMRMTokens | |
| hadoop.yarn.server.resourcemanager.TestAppManager | |
| hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption | |
| hadoop.yarn.server.resourcemanager.TestApplicationMasterService | |
| hadoop.yarn.server.resourcemanager.TestClientRMService | |
| hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification | |
| hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus | |
| hadoop.yarn.server.resourcemanager.TestRMHA | |
| hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue | |
| hadoop.yarn.server.resourcemanager.TestRMRestart | |
| hadoop.yarn.server.resourcemanager.rmapp.attempt.TestRMAppAttemptTransitions | |
| hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter | |
| hadoop.yarn.server.resourcemanager.TestRMAdminService | |
| hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler | |
| hadoop.yarn.server.resourcemanager.TestApplicationCleanup | |
| hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | |
| hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart | |
| hadoop.yarn.server.resourcemanager.TestAMAuthorization | |
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps |
| org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 17m 34s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 7 new or modified test files. |
| +1 | javac | 7m 41s | There were no new javac warning messages. |
| +1 | javadoc | 9m 38s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 49s | The applied patch generated 7 new checkstyle issues (total was 87, now 88). |
| -1 | whitespace | 0m 1s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 21s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 3m 3s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | yarn tests | 1m 56s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 52m 50s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 96m 52s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.yarn.util.TestRackResolver |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerEventLog | |
| hadoop.yarn.server.resourcemanager.TestRMHA | |
| hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | |
| hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing | |
| hadoop.yarn.server.resourcemanager.rmapp.attempt.TestRMAppAttemptTransitions | |
| hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions | |
| hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels | |
| hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector | |
| hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA | |
| hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations | |
| hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes | |
| hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus | |
| hadoop.yarn.server.resourcemanager.TestRMAdminService | |
| hadoop.yarn.server.resourcemanager.TestResourceManager | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue | |
| hadoop.yarn.server.resourcemanager.TestAppManager |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| -1 | pre-patch | 16m 1s | Findbugs (version ) appears to be broken on trunk. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 7 new or modified test files. |
| +1 | javac | 7m 48s | There were no new javac warning messages. |
| +1 | javadoc | 9m 42s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 20s | The applied patch does not increase the total number of release audit warnings. |
| +1 | checkstyle | 1m 30s | There were no new checkstyle issues. |
| -1 | whitespace | 0m 1s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 24s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 31s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 4m 50s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | common tests | 22m 24s | Tests failed in hadoop-common. |
| -1 | yarn tests | 1m 54s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 52m 20s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 118m 48s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.net.TestNetUtils |
| hadoop.security.token.delegation.web.TestWebDelegationToken | |
| hadoop.ha.TestZKFailoverController | |
| hadoop.yarn.util.TestRackResolver | |
| hadoop.yarn.server.resourcemanager.rmapp.attempt.TestRMAppAttemptTransitions | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue | |
| hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | |
| hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions | |
| hadoop.yarn.server.resourcemanager.logaggregationstatus.TestRMAppLogAggregationStatus | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations | |
| hadoop.yarn.server.resourcemanager.TestResourceManager | |
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestRMHA |
This message was automatically generated.
thanks Jian He for the patch, I have some doubts/comments on the patch..
If this time becomes larger than 10 mins, all applications will expire
For expire , the new events would trigger which is blocked since dispatcher is draining. Am I missing any specific scenario which can occure?
- Why the end time considering half of the rm-expire interval? just to know any feasibility reason.
- Proceding to NM-RM expire, now gracefull shutdown of NM is also supported. So IIUC NM expire can also trigger if draining is taking more than 10 minutes. Thinking any impact on unregister nodemanager? Now timeout is based on the AM_RM expire config, IAC, should NM-RM expire also considered?any thoughts?
- Ther is issue exist for test failures in TestRMAdminService, so I think changes for JvmPauseMonitor.java can be tracked in separate jira i.e.
YARN-4035. What do you think?
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 17m 28s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 8 new or modified test files. |
| +1 | javac | 7m 41s | There were no new javac warning messages. |
| +1 | javadoc | 9m 45s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 1m 41s | The applied patch generated 7 new checkstyle issues (total was 87, now 88). |
| -1 | whitespace | 0m 2s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 20s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 3m 2s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | yarn tests | 1m 55s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 52m 19s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 96m 12s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.yarn.util.TestRackResolver |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerEventLog | |
| hadoop.yarn.server.resourcemanager.TestRMHA | |
| hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | |
| hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing | |
| hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels | |
| hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector | |
| hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA | |
| hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | |
| hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes | |
| hadoop.yarn.server.resourcemanager.TestRMAdminService | |
| hadoop.yarn.server.resourcemanager.TestResourceManager |
This message was automatically generated.
Thanks for reviewing the patch. Rohith.
For expire , the new events would trigger which is blocked since dispatcher is draining.
ATS events processing happens on a different dispatcher.
Why the end time considering half of the rm-expire interval
No strong reason why it depends on am-rm-expiry. Just want to avoid introducing rarely-used config. Anyway, I introduced a new config for this.
Ther is issue exist for test failures in TestRMAdminService.
I would like to introduce this fix to trigger jenkins. Otherwise, Too many failures. I can't see a clean results.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 21m 14s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 8 new or modified test files. |
| +1 | javac | 9m 56s | There were no new javac warning messages. |
| +1 | javadoc | 12m 3s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 26s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 3m 24s | The applied patch generated 1 new checkstyle issues (total was 211, now 211). |
| -1 | checkstyle | 4m 12s | The applied patch generated 7 new checkstyle issues (total was 87, now 88). |
| -1 | whitespace | 0m 2s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 38s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 8m 41s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | common tests | 26m 9s | Tests failed in hadoop-common. |
| -1 | yarn tests | 0m 26s | Tests failed in hadoop-yarn-api. |
| -1 | yarn tests | 2m 23s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 54m 24s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 142m 14s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.crypto.key.TestValueQueue |
| hadoop.ha.TestZKFailoverController | |
| hadoop.fs.TestLocalFsFCStatistics | |
| hadoop.net.TestNetUtils | |
| hadoop.yarn.conf.TestYarnConfigurationFields | |
| hadoop.yarn.util.TestRackResolver | |
| hadoop.yarn.event.TestAsyncDispatcher | |
| hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | |
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
| org.apache.hadoop.yarn.server.resourcemanager.TestRMHA |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 20m 54s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 8 new or modified test files. |
| +1 | javac | 7m 48s | There were no new javac warning messages. |
| +1 | javadoc | 9m 41s | There were no new javadoc warning messages. |
| +1 | release audit | 0m 21s | The applied patch does not increase the total number of release audit warnings. |
| -1 | checkstyle | 3m 2s | The applied patch generated 1 new checkstyle issues (total was 211, now 211). |
| -1 | checkstyle | 3m 40s | The applied patch generated 7 new checkstyle issues (total was 87, now 88). |
| -1 | whitespace | 0m 2s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 23s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 31s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 6m 20s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| -1 | common tests | 22m 17s | Tests failed in hadoop-common. |
| +1 | yarn tests | 0m 22s | Tests passed in hadoop-yarn-api. |
| -1 | yarn tests | 1m 54s | Tests failed in hadoop-yarn-common. |
| -1 | yarn tests | 53m 9s | Tests failed in hadoop-yarn-server-resourcemanager. |
| 128m 26s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.net.TestNetUtils |
| hadoop.ha.TestZKFailoverController | |
| hadoop.yarn.util.TestRackResolver | |
| hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
This message was automatically generated.
Thanks Jian He for updating the patch..
One doubt is SystemMetricsPublisher has been moved from RMActiveServices to ResourceManager. So this service will not be reinitialized on every RM switch. Thinking that this would lead for processing stale events even after RM is in standby. If any case, the same RM becomes active SystemMetricsPublisher dispatcher publishes stale events plus recovered application events. Anyway events processing will happen in the sequential order if same RM comes back Active. But issue may can ocure when the different RM becomes active i.e
- RM1 is active and publishing the events
- RM1 is transitioning to standby,and some events are in the queue to be updated in the timeline sever
- RM2 become active and recovered the applications. When application got finished, RM2 systempublisher publishes app status as finished.
- RM1 is still processing the events for app which would process bit late i.e after RM2 processed.
Doesn't it cause problem? Any thoughts?
I talked to Zhijie Shen about this too. I think it's fine as the event processing order is not that critical. Also each timeline entity has a timestamp which itself indicates the order of the event too.IMO, this is similar to multiple containers writing to ATS at the same time. There's no guarantee that the earliest generated event gets published into ATS first.
thank Jian He for the explanation. Overall patch looks good to me..
+1 lgtm. Will commit later if there are no other comments
Thanks, Jian. Committed into trunk/branch-2/branch-2.7.
FAILURE: Integrated in Hadoop-trunk-Commit #8286 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8286/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
FAILURE: Integrated in Hadoop-Yarn-trunk #1015 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1015/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #285 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/285/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
FAILURE: Integrated in Hadoop-Hdfs-trunk #2212 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2212/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #282 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/282/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2231 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2231/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/CHANGES.txt
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #274 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/274/)
YARN-3999. RM hangs on draing events. Contributed by Jian He (xgong: rev 3ae716fa696b87e849dae40225dc59fb5ed114cb)
- hadoop-yarn-project/CHANGES.txt
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/logaggregationstatus/TestRMAppLogAggregationStatus.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/event/TestAsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java
- hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java
Adding to 2.6.1 from Jian's comment in the mailing list that I missed before.
Pulled this into 2.6.1.
Had to fix a couple of minor merge conflicts. Dropped changes to TestAsyncDispatcher.java and TestRMAppLogAggregationStatus.java which don't exist in 2.6.1.
Ran compilation and TestAppManager, TestResourceManager, TestRMAppTransitions, TestRMAppAttemptTransitions, TestUtils, TestFifoScheduler before the push.
Attaching patch that I committed to 2.6.1.
Uploaded a patch which adds the timeout on draining the dispatcher. The value is set to be half of the am-rm-expiry-time.
Beyond that, I also changed the order of a couple of services which might take long time to flush the events on stop.