[HIVE-18946] Fix columnstats merge NPE

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0, 3.0.0
• Target Version/s: 3.0.0
• Assignee: László Bodor Assign to me
• Reporter: Zoltan Haindrich
• Created: 13/Mar/18 17:13
• Updated: 23/May/18 02:16
• Resolved: 17/Apr/18 12:08

Description

after analyzing an empty table may lead to an NPE when inserting into it...

2018-03-13T06:54:22,503 ERROR [df3fb505-e0bc-4595-a874-b735dab8dff6 main] metastore.RetryingHMSHandler: java.lang.NullPointerException
        at org.apache.hadoop.hive.metastore.api.Decimal.compareTo(Decimal.java:318)
        at org.apache.hadoop.hive.metastore.columnstats.merge.DecimalColumnStatsMerger.merge(DecimalColumnStatsMerger.java:35)
        at org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.mergeColStats(MetaStoreUtils.java:778)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.set_aggr_stats_for(HiveMetaStore.java:6934)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
        at com.sun.proxy.$Proxy55.set_aggr_stats_for(Unknown Source)
[...]

reproduce

set hive.stats.autogather=true;
set hive.explain.user=true;

drop table if exists testdeci2;

create table testdeci2(
id int,
amount decimal(10,3),
sales_tax decimal(10,3),
item string)
stored as orc location '/tmp/testdeci2'
TBLPROPERTIES ("transactional"="false")
;


analyze table testdeci2 compute statistics for columns;

insert into table testdeci2 values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2');

Issue Links

• duplicates: HIVE-19160 Insert data into decimal column fails with Null Pointer Exception
• is related to: HIVE-19131 DecimalColumnStatsMergerTest comparison review