Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
Currently we are not handling the case of stage cancellation and if the stage is cancelled while the stage is about to get merge finalized.
Stage completion will wait for shuffle merge finalize wait time so that all the shuffle merge requests are drained but during this time the stage can be cancelled, if this happens then we need to let all the shuffle services to remove the merged shuffle data for the corresponding shuffle ID.