[ORC-134] DecimalColumnStatistics methods throws NPE when all column values are null

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.1, 1.4.0
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 23/Jan/17 21:54
• Updated: 03/Feb/17 15:54
• Resolved: 24/Jan/17 00:08

Description

FastDecimal changes introduced a regression that when all column column values in stripe or file is null, the min, max values are also null. getMinimum() and getMaximum() methods in ColumnStatisticsImpl is throwing NPE. Following is the stacktrace

Caused by: java.lang.NullPointerException
	at org.apache.orc.impl.ColumnStatisticsImpl$DecimalStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:673)
	at org.apache.orc.impl.RecordReaderImpl.getMin(RecordReaderImpl.java:322)
	at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:368)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2088)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2067)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2058)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:149)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1376)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1164)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1345)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1342)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1342)
	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1164)

Issue Links

• links to: GitHub Pull Request #86