[MAPREDUCE-6831] Flaky test TestJobImpl.testKilledDuringKillAbort

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha2, 2.8.2
• Component/s: mrv2
• Hadoop Flags: Reviewed
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 09/Jan/17 15:19
• Updated: 07/Jun/17 22:29
• Resolved: 17/Jan/17 16:06

Description

The test case TestJobImpl.testKilledDuringKillAbort() is flaky.

Example of a failure:

Error Message

expected:<SETUP> but was:<RUNNING>

Stack Trace

java.lang.AssertionError: expected:<SETUP> but was:<RUNNING>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:978)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testKilledDuringKillAbort(TestJobImpl.java:516)

Standard Output

2016-12-12 00:26:29,724 INFO  [Thread-12] event.AsyncDispatcher (AsyncDispatcher.java:register(202)) - Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2016-12-12 00:26:29,729 INFO  [Thread-12] event.AsyncDispatcher (AsyncDispatcher.java:register(202)) - Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl$StubbedJob
2016-12-12 00:26:29,729 INFO  [Thread-12] event.AsyncDispatcher (AsyncDispatcher.java:register(202)) - Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.yarn.event.EventHandler$$EnhancerByMockitoWithCGLIB$$2a4993a5
2016-12-12 00:26:29,730 INFO  [Thread-12] event.AsyncDispatcher (AsyncDispatcher.java:register(202)) - Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.yarn.event.EventHandler$$EnhancerByMockitoWithCGLIB$$2a4993a5
2016-12-12 00:26:29,730 INFO  [Thread-12] event.AsyncDispatcher (AsyncDispatcher.java:register(202)) - Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.yarn.event.EventHandler$$EnhancerByMockitoWithCGLIB$$2a4993a5
2016-12-12 00:26:29,730 INFO  [Thread-12] impl.JobImpl (JobImpl.java:setup(1523)) - Adding job token for job_1234567890000_0001 to jobTokenSecretManager
2016-12-12 00:26:29,731 WARN  [Thread-12] impl.JobImpl (JobImpl.java:setup(1529)) - Shuffle secret key missing from job credentials. Using job token secret as shuffle secret.
2016-12-12 00:26:29,733 INFO  [Thread-12] impl.JobImpl (JobImpl.java:makeUberDecision(1294)) - Not uberizing job_1234567890000_0001 because: not enabled;
2016-12-12 00:26:29,734 INFO  [Thread-12] impl.JobImpl (JobImpl.java:createMapTasks(1551)) - Input size for job job_1234567890000_0001 = 0. Number of splits = 2
2016-12-12 00:26:29,734 INFO  [Thread-12] impl.JobImpl (JobImpl.java:createReduceTasks(1568)) - Number of reduces for job job_1234567890000_0001 = 1
2016-12-12 00:26:29,734 INFO  [Thread-12] impl.JobImpl (JobImpl.java:handle(1006)) - job_1234567890000_0001Job Transitioned from NEW to INITED
2016-12-12 00:26:29,736 INFO  [CommitterEvent Processor #0] commit.CommitterEventHandler (CommitterEventHandler.java:run(231)) - Processing the event EventType: JOB_SETUP
2016-12-12 00:26:29,737 INFO  [Thread-12] impl.JobImpl (JobImpl.java:handle(1006)) - job_1234567890000_0001Job Transitioned from INITED to SETUP
2016-12-12 00:26:29,738 INFO  [AsyncDispatcher event handler] impl.JobImpl (JobImpl.java:handle(1006)) - job_1234567890000_0001Job Transitioned from SETUP to RUNNING

Reproduction: insert a Thread.sleep(50); after job.handle(new JobStartEvent(jobId));