Description
CaseKeyWhen implementation in Spark uses the following equals implementation:
private def equalNullSafe(l: Any, r: Any) = { if (l == null && r == null) { true } else if (l == null || r == null) { false } else { l == r } }
Which is not correct, since in SQL, NULL is never equal to NULL (actually, it is not unequal either). In this case, a NULL value in a CASE WHEN expression should never match.
For example, you can execute this in MySQL:
SELECT CASE NULL WHEN NULL THEN "NULL MATCHES" ELSE "NULL DOES NOT MATCH" END FROM DUAL;
And the result will be "NULL DOES NOT MATCH".
Attachments
Issue Links
- is related to
-
SPARK-3813 Support "case when" conditional functions in Spark SQL
-
- Resolved
-
- links to
