[HIVE-13423] Handle the overflow case for decimal datatype for sum()

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.2.0
• Component/s: Query Processor
• Assignee: Aihua Xu
• Reporter: Aihua Xu
• Created: 05/Apr/16 13:49
• Updated: 26/Jul/17 03:30
• Resolved: 20/Oct/16 13:40

Description

When a column col1 defined as decimal and if the sum of the column overflows, we will try to increase the decimal precision by 10. But if it's reaching 38 (the max precision), the overflow still could happen. Right now, if such case happens, the following exception will throw since hive is writing incorrect data.

Follow the following steps to repro.

CREATE TABLE DECIMAL_PRECISION(dec decimal(38,18));
INSERT INTO DECIMAL_PRECISION VALUES(98765432109876543210.12345), (98765432109876543210.12345);
SELECT SUM(dec) FROM DECIMAL_PRECISION;

Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:314) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:219) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]

Issue Links

• relates to: HIVE-6459 Change the precison/scale for intermediate sum result in the avg() udf