[IMPALA-6661] Group by float results in one group per NaN value

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.11.0, Impala 2.12.0
• Fix Version/s: Impala 3.1.0
• Component/s: Backend
• Labels: correctness perf ramp-up
• Epic Color: ghx-label-9
• Assignee: Michal Ostrowski
• Reporter: Tim Armstrong
• Created: 14/Mar/18 16:30
• Updated: 02/Nov/18 15:23
• Resolved: 02/Nov/18 15:20

Description

I don't know if this is the desired behaviour but it could be problematic for some users since it will blow up the number of distinct groups in an aggregation. I suspect that it's more useful to coalesce all the NaNs into a single group, similar to how NULL is handled in GROUP BY.

[localhost:21000] > select distinct * from (values(cast("nan" as float)), (cast("nan" as float)), (sqrt(cast("-1" as float)))) v;
+----------------------+
| cast('nan' as float) |
+----------------------+
| NaN                  |
| NaN                  |
| NaN                  |
+----------------------+
Fetched 3 row(s) in 0.11s

I suspect IMPALA-6069 slightly changed the behaviour here, although it would have been broken beforehand anyway, since not all NaNs have the same bit pattern, so Equals() and Hash() were inconsistent.

We should decided what the preferred behaviour is and tweak the behaviour of the hash table to produce it.

Issue Links

• relates to: IMPALA-6069 Incorrect handling of Nan with join and codegen
• relates to: IMPALA-6660 -0/+0 floating point do not compare as equal in hash table
• relates to: IMPALA-1543 Positive and negative zero floats hash and compare as unequal, although they should be equal.