[SPARK-28237] Idempotence checker for Idempotent batches in RuleExecutors

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Yesheng Ma
• Reporter: Yesheng Ma
• Created: 02/Jul/19 23:38
• Updated: 28/Jul/19 21:33
• Resolved: 26/Jul/19 06:45

Description

The current RuleExecutor system contains two kinds of strategies: Once and FixedPoint. The Once strategy is supposed to run once. However, for particular rules (e.g. PullOutNondeterministic), they are designed to be idempotent, but Spark currently lacks corresponding mechanism to prevent such kind of non-idempotent behavior from happening.

Issue Links

• links to: GitHub Pull Request #25249