[TEZ-2022] java.lang.IllegalStateException: Vertex: got invalid start event

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Target Version/s: 0.8.3
• Assignee: Jeff Zhang
• Reporter: Rajesh Balamohan
• Created: 02/Feb/15 09:43
• Updated: 17/Mar/16 07:47
• Resolved: 17/Mar/16 07:47

Description

Latest Tez (with tez-2020 patch) + Pig

When running pig with rank function, the following exception happens consistently

java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03 [scope-47] got invalid start event
        at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
        at org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
        at org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
        at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
        at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
        at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
        at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
        at java.lang.Thread.run(Thread.java:744)

rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray, line:chararray);
raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1)) as machine, key, line;
machines = FOREACH raw GENERATE machine;
distinctMachines = DISTINCT machines;
sortByMachines = ORDER distinctMachines BY machine;
ranked = RANK sortByMachines;
dump ranked;

Will attach the DAG and applog asap.