[IMPALA-11323] Invalid inferred predicates based on casted null values being equivalent

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0, Impala 4.1.0
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Component/s: Frontend
• Labels: correctness
• Epic Color: ghx-label-9
• Assignee: Steve Carlin
• Reporter: Joe McDonnell
• Created: 27/May/22 00:19
• Updated: 23/Aug/22 05:51
• Resolved: 07/Jun/22 13:17

Description

This query should return two rows, but instead it returns nothing:

[localhost:21050] default> with t as (select 1 a), v as (select distinct a, cast(null as smallint)b, cast(null as smallint)c from t) select distinct a,b,c from v union all select distinct a,b,c from v;

Fetched 0 row(s) in 0.14s

Without the union all, each side is producing a single row, so it stands to reason that the union all would produce two rows:

[localhost:21050] default> with t as (select 1 a), v as (select distinct a, cast(null as smallint)b, cast(null as smallint)c from t) select distinct a,b,c from v;

+---+------+------+
| a | b    | c    |
+---+------+------+
| 1 | NULL | NULL |
+---+------+------+
Fetched 1 row(s) in 0.14s

Both b and c are hard coded to null in the definition of v. The plan shows that it is producing a b = c predicate:

+-------------------------------------------------------------------+
| Explain String                                                    |
+-------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=7.88MB Threads=1        |
| Per-Host Resource Estimates: Memory=24MB                          |
| Codegen disabled by planner                                       |
|                                                                   |
| PLAN-ROOT SINK                                                    |
| |                                                                 |
| 00:UNION                                                          |
| |  pass-through-operands: all                                     |
| |  row-size=5B cardinality=2                                      |
| |                                                                 |
| |--08:SELECT                                                      |
| |  |  predicates: b = c                                           |
| |  |  row-size=5B cardinality=1                                   |
| |  |                                                              |
| |  07:AGGREGATE [FINALIZE]                                        |
| |  |  group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT) |
| |  |  row-size=5B cardinality=1                                   |
| |  |                                                              |
| |  06:AGGREGATE [FINALIZE]                                        |
| |  |  group by: a, CAST(NULL AS SMALLINT)                         |
| |  |  row-size=3B cardinality=1                                   |
| |  |                                                              |
| |  05:UNION                                                       |
| |     constant-operands=1                                         |
| |     row-size=1B cardinality=1                                   |
| |                                                                 |
| 04:SELECT                                                         |
| |  predicates: b = c                                              |
| |  row-size=5B cardinality=1                                      |
| |                                                                 |
| 03:AGGREGATE [FINALIZE]                                           |
| |  group by: a, CAST(NULL AS SMALLINT), CAST(NULL AS SMALLINT)    |
| |  row-size=5B cardinality=1                                      |
| |                                                                 |
| 02:AGGREGATE [FINALIZE]                                           |
| |  group by: a, CAST(NULL AS SMALLINT)                            |
| |  row-size=3B cardinality=1                                      |
| |                                                                 |
| 01:UNION                                                          |
|    constant-operands=1                                            |
|    row-size=1B cardinality=1                                      |
+-------------------------------------------------------------------+

I added tracing to PlanNode's addConjunctsToNode() and the b=c conjunct is generated by Analyzer::createEquivConjuncts(). It survives the inferred identity predicate checks, because it is referring to two different slots.