[MAPREDUCE-4748] Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.23.3
• Fix Version/s: 2.0.3-alpha, 0.23.5
• Component/s: mrv2
• Target Version/s: 2.0.3-alpha, 0.23.5
• Hadoop Flags: Reviewed
• Assignee: Jason Darrell Lowe
• Reporter: Robert Joseph Evans
• Created: 24/Oct/12 21:53
• Updated: 06/Feb/13 17:05
• Resolved: 26/Oct/12 21:08

Description

We saw this happen when running a large pig script.

2012-10-23 22:45:24,986 ERROR [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Can't handle this event at current state for task_1350837501057_21978_m_040453
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: T_ATTEMPT_SUCCEEDED at SUCCEEDED
        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
        at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.handle(TaskImpl.java:604)
        at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.handle(TaskImpl.java:89)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher.handle(MRAppMaster.java:914)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher.handle(MRAppMaster.java:908)
        at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
        at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
        at java.lang.Thread.run(Thread.java:619)

Speculative execution was enabled, and that task did speculate so it looks like this is an error in the state machine either between the task attempts or just within that single task.

Issue Links

• blocks: MAPREDUCE-4751 AM stuck in KILL_WAIT for days