Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
Clone tasks immediately, without waiting for anything. (e.g., waiting until some other tasks finish to identify stragglers)
Tests
- TaskRetryTest: Cloned tasks should be handled correctly under failures
- WordCountITCase#testClones: Cloned Map and Reduce tasks
Optimizer
- CloneProperty extends VertexExecutionProperty<Integer>
- ClonePass: Sets CloneProperty to 2
Runtime
- A state machine per task attempt
- Data Plane: Use any one of the parent task attempts' output blocks
- Control Plane: Looks up the CloneProperty to schedule N attempts at a time