[SPARK-28068] `lag` second argument must be a literal

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Dylan Guedes
• Created: 16/Jun/19 21:35
• Updated: 29/Mar/20 02:31

Description

Currently in Spark, `lag` (and, possible, some other window functions) requires the 2nd argument to be a literal.
For example, this is not allowed:

SELECT lag(ten, four) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10;

However, this one works:

SELECT lag(ten, 2) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10;

In comparison, Postgres accepts a literal as a 2nd argument. I found this issue while porting `window.sql` tests from Postgres to Spark