[HIVE-24908] Adding Respect/Ignore nulls as a UDAF parameter is ambiguous

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: UDF
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 19/Mar/21 10:58
• Updated: 17/Nov/22 08:47
• Resolved: 23/Mar/21 05:32

Description

Both function calls translated to the same UDAF call:

SELECT lead(a, 2, true) ...
SELECT lead(a, 2) IGNORE NULLS ...

IGNORE NULLS is passed as an extra constant boolean parameter to the UDAF
https://github.com/apache/hive/blob/eed78dfdcb6dfc2de400397a60de12e6f62b96e2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java#L743

However the semantics of the two function calls has different semantics:

• lead(a, 2, true) - 'true' is the default value: "The value of DEFAULT is returned as the result if there is no row corresponding to the OFFSET number of rows before R within P (for the lag function) or after R within P (for the lead function)"
• lead(a, 2) IGNORE NULLS - For each row in the current window find the 2nd not-NULL value starting directly after the current row.

Issue Links

• is related to: HIVE-24865 Implement Respect/Ignore Nulls in first/last_value
• links to: GitHub Pull Request #2091