[SPARK-25247] Make RDDBarrier configurable

• Type: Story
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.0
• Component/s: Spark Core
• Labels: bulk-closed
• Epic Link: Support Barrier Execution Mode
• Assignee: Unassigned
• Reporter: Xiangrui Meng
• Created: 27/Aug/18 03:46
• Updated: 25/May/21 01:55
• Resolved: 25/May/21 01:39

Description

Currently we only offer one method under `RDDBarrier`. Users might want to have better control over a barrier stage, e.g., timeout behavior, failure recovery, etc. This Jira is to discuss what options we should provide under RDDBarrier.

 

Note: Users can use multiple RDDBarrier in a single barrier stage. So we also need to discuss how to merge the options.