[SPARK-22895] Push down the deterministic predicates that are after the first non-deterministic

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.1
• Fix Version/s: 2.3.0
• Component/s: SQL
• Assignee: Xiao Li
• Reporter: Xiao Li
• Created: 24/Dec/17 06:29
• Updated: 31/Dec/17 07:07
• Resolved: 31/Dec/17 07:07

Description

Currently, we do not guarantee an order evaluation of conjuncts in either Filter or Join operator. This is also true to the mainstream RDBMS vendors like DB2 and MS SQL Server. Thus, we should also push down the deterministic predicates that are after the first non-deterministic, if possible.

Issue Links

• links to: [Github] Pull Request #20069 (gatorsmile)