[ORC-1146] Float category missing check if the statistic sum is a finite value

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.3
• Fix Version/s: 1.7.4, 1.6.14
• Component/s: Java
• Assignee: Yiqun Zhang
• Reporter: Yiqun Zhang
• Created: 04/Apr/22 13:15
• Updated: 15/Apr/22 23:26
• Resolved: 04/Apr/22 17:41

Description

https://github.com/apache/orc/blob/70c504cbd759342476d2fb2c9cc75a5025cb7c11/java/core/src/java/org/apache/orc/impl/RecordReaderImpl.java#L696-L703

} else if (category == TypeDescription.Category.DOUBLE) { 
   DoubleColumnStatistics dstas = (DoubleColumnStatistics) cs; 
   if (!Double.isFinite(dstas.getSum())) { 
     LOG.debug("Not using predication pushdown on {} because stats contain NaN values", 
             predicate.getColumnName()); 
     return dstas.hasNull() ? TruthValue.YES_NO_NULL : TruthValue.YES_NO; 
   } 
 } 

Currently only the Double category is checked. the Float category is also only pushed down if the statistics sum to a finite value.

Issue Links

• links to: GitHub Pull Request #1077
• links to: GitHub Pull Request #1078