Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Checkpoint key contains task name and that task name is used for last processed offsets when container starts up. However, with elasticity task names change as follows from "Partition 0" to "Partition 0_1_2" (0 is the input partition, 1 is the key bucket and 2 is the elasticity factor). Thus when elasticity is enabled then computing last processed offsets need to take into account both Partition 0 and Partition 0_1_2 when computing checkpoint for either.
This ticket is to update the OffsetManager to do this computation for elasticity
