[TEZ-1015] Dag failed with Invalid event: V_ROUTE_EVENT at RECOVERING

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0, 0.4.1
• Assignee: Unassigned
• Reporter: Arpit Gupta
• Created: 03/Apr/14 21:02
• Updated: 06/Sep/14 01:35
• Resolved: 04/Apr/14 21:58

Description

During HA tests hive job was run where NN was killed in the background and RM was also killed once. The job failed with

2014-04-03 08:59:03,253 ERROR [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.VertexImpl: Can't handle Invalid event V_ROUTE_EVENT on vertex Reducer 8 with vertexId vertex_1396515360091_0001_1_03 at current state RECOVERING
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_ROUTE_EVENT at RECOVERING
	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
	at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1210)
	at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:156)
	at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1659)
	at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1645)
	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
	at java.lang.Thread.run(Thread.java:662)

Issue Links

• is related to: TEZ-850 Recovery unit tests
• relates to: TEZ-1019 Re-factor routing of events to use common code path for normal and recovery flow.