[SPARK-31651] Improve handling the case where different barrier sync types in a single sync

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Assignee: wuyi
• Reporter: wuyi
• Created: 06/May/20 09:34
• Updated: 19/May/20 07:19
• Resolved: 19/May/20 07:18

Description

Currently, we use cleanupBarrierStage when detecting different barrier sync types in a single sync. This cause a problem that a new `ContextBarrierState` can be created again if there's following requesters on the way. And those corresponding tasks will fail because of killing instead of different barrier sync types detected.

 

 

Issue Links

• links to: [Github] Pull Request #28462 (Ngone51)