[SPARK-28553] subqueries must be aggregated before hand

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Dylan Guedes
• Created: 29/Jul/19 11:53
• Updated: 30/Jul/19 12:09
• Resolved: 30/Jul/19 12:09

Description

Looks like Spark subqueries does not work well with variables and values from outside of subquery. For instance, this query work on PgSQL:
 

SELECT lead(ten, (SELECT two FROM tenk1 WHERE s.unique2 = unique2)) OVER (PARTITION BY four ORDER BY ten) FROM tenk1 s WHERE unique2 < 10;

However, it does not work in Spark.