[TEZ-2074] TaskRescheduledAfterVertexSuccessTransition may go back to RUNNING incorrectly

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Bikas Saha
• Created: 11/Feb/15 02:20
• Updated: 11/Feb/15 03:02

Description

    public VertexState transition(VertexImpl vertex, VertexEvent event) {
      if (vertex.outputCommitters == null // no committer
          || vertex.outputCommitters.isEmpty() // no committer
          || !vertex.commitVertexOutputs) { // committer does not commit on vertex success
        LOG.info(vertex.getLogIdentifier() + " back to running due to rescheduling "
            + ((VertexEventTaskReschedule)event).getTaskID());
        (new TaskRescheduledTransition()).transition(vertex, event);
        // inform the DAG that we are re-running
        vertex.eventHandler.handle(new DAGEventVertexReRunning(vertex.getVertexId()));
        return VertexState.RUNNING;
      }
      ...
    }
 

The "// committer does not commit on vertex success" may be wrong because the DAG might have completed at this time and the overall commit might be in progress. If so, the Vertex and DAG should fail.