[TEZ-2404] Handle DataMovementEvent before its TaskAttemptCompletedEvent

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Target Version/s: 0.7.0
• Hadoop Flags: Reviewed
• Assignee: Jeff Zhang
• Reporter: Jeff Zhang
• Created: 04/May/15 03:42
• Updated: 30/Jun/15 04:52
• Resolved: 07/May/15 18:46

Description

TEZ-2325 route TASK_ATTEMPT_COMPLETED_EVENT directly to the attempt, but it would cause recovery issue. Recovery need that DataMovement event is handled before TaskAttemptCompletedEvent, otherwise DataMovement event may be lost in recovering and cause the its dependent tasks hang.

2 Ways to fix this issue.

1. Still route TaskAtttemptCompletedEvent in Vertex
2. route DataMovementEvent before TaskAttemptCompeltedEvent in TezTaskAttemptListener

Issue Links

• breaks: TEZ-2418 TASK_ATTEMPT_FAILED_EVENT and TASK_COMPLETED_EVENT should move back to direct routing to attempt
• duplicates: TEZ-2402 Handle DataMovementEvent before its TaskAttemptCompletedEvent
• duplicates: TEZ-2403 Handle DataMovementEvent before its TaskAttemptCompletedEvent
• is related to: TEZ-1897 Create a concurrent version of AsyncDispatcher