[SPARK-17244] Joins should not pushdown non-deterministic conditions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: SQL
• Labels: correctness
• Assignee: Sameer Agarwal
• Reporter: Sameer Agarwal
• Created: 25/Aug/16 20:49
• Updated: 12/Dec/17 06:26
• Resolved: 26/Aug/16 23:41

Description

Given that non-deterministic expressions can be stateful, pushing them down the query plan during the optimization phase can cause incorrect behavior.

Issue Links

• links to: [Github] Pull Request #14815 (sameeragarwal)