[IMPALA-865] Impala will return incorrect result on int overflow without warning

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 1.3
• Component/s: Backend
• Labels: correctness downgraded
• Epic Link: Numeric Datatype Conversions
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: casey
• Created: 10/Mar/14 20:18
• Updated: 10/Apr/18 22:48

Description

A query that has a select item that results in an int greater than the max bigint value will be incorrect and no warning will be given to the user.

It would be great if Impala could automatically convert the int to a float or double (which is what MySQL does) or at least warn that the value is incorrect.

Here is an example from the random query generator. The bottom query has "1.00 * " prepended to force a float/double. This is a crazy query but the same problem could come up on a simpler query if the dataset was large enough.

[localhost:21000] > select MIN(t2.tinyint_col_2 - t2.int_col_6 - t1.int_col_9) * SUM(t2.tinyint_col_2) * COUNT(t1.bigint_col_7) * SUM(t2.smallint_col_7 * t2.smallint_col_9) - COUNT(t2.bigint_col_5) AS col_1 FROM table_4 AS t1 LEFT JOIN table_2 AS t2 ON t2.bigint_col_10 = t1.smallint_col_4;
Query: select MIN(t2.tinyint_col_2 - t2.int_col_6 - t1.int_col_9) * SUM(t2.tinyint_col_2) * COUNT(t1.bigint_col_7) * SUM(t2.smallint_col_7 * t2.smallint_col_9) - COUNT(t2.bigint_col_5) AS col_1 FROM table_4 AS t1 LEFT JOIN table_2 AS t2 ON t2.bigint_col_10 = t1.smallint_col_4
+--------------------+
| col_1              |
+--------------------+
| 581080169910184334 |
+--------------------+
Returned 1 row(s) in 0.14s


[localhost:21000] > select 1.00* MIN(t2.tinyint_col_2 - t2.int_col_6 - t1.int_col_9) * SUM(t2.tinyint_col_2) * COUNT(t1.bigint_col_7) * SUM(t2.smallint_col_7 * t2.smallint_col_9) - COUNT(t2.bigint_col_5) AS col_1 FROM table_4 AS t1 LEFT JOIN table_2 AS t2 ON t2.bigint_col_10 = t1.smallint_col_4;
Query: select 1.00* MIN(t2.tinyint_col_2 - t2.int_col_6 - t1.int_col_9) * SUM(t2.tinyint_col_2) * COUNT(t1.bigint_col_7) * SUM(t2.smallint_col_7 * t2.smallint_col_9) - COUNT(t2.bigint_col_5) AS col_1 FROM table_4 AS t1 LEFT JOIN table_2 AS t2 ON t2.bigint_col_10 = t1.smallint_col_4
+------------------------+
| col_1                  |
+------------------------+
| -3.130135690831522e+20 |
+------------------------+
Returned 1 row(s) in 0.14s

Issue Links

• is related to: IMPALA-2710 Overflows during expr evaluation should result in an error or null
• is related to: IMPALA-2633 Inconsistent handling of out-of-range values in integer casts
• relates to: IMPALA-6072 Uncaught overflow when multiplying two bigints
• relates to: IMPALA-2020 Rounding should be done instead of truncating when casting DECIMAL to DECIMAL, FLOAT/DOUBLE to DECIMAL, DECIMAL to INT