[SPARK-14785] Support correlated scalar subquery

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Epic Link: Support all TPCDS queries
• Assignee: Herman van Hövell
• Reporter: Davies Liu
• Created: 20/Apr/16 23:57
• Updated: 02/May/16 23:33
• Resolved: 02/May/16 23:32

Description

For example:

SELECT a from t where b > (select avg(c) from t2 where t.id = t2.id)

it could be rewritten as

SELECT a FROM t JOIN (SELECT id, AVG(c) as avg_c FROM t2 GROUP by id) t3 ON t3.id = t.id where b > avg_c

TPCDS Q92, Q81, Q6 required this

Update: TPCDS Q1 and Q30 also require correlated scalar subquery support.

Issue Links

• is duplicated by: SPARK-14968 TPC-DS query 1 resolved attribute(s) missing
• links to: [Github] Pull Request #12815 (hvanhovell)
• links to: [Github] Pull Request #12822 (hvanhovell)