[ORC-500] PPD: StripeStatistics::getColumnStatistics() does not column prune

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.5.5
• Component/s: Java
• Assignee: Unassigned
• Reporter: Gopal Vijayaraghavan
• Created: 16/May/19 00:30
• Updated: 16/May/19 01:17

Description

StripeStatistics::getColumnStatistics() will deserialize all column statistics, even if the ORC predicate push-down and the eventual SARG applies to a subset.

This is affecting DATE push-down into a very wide table, where the deserialization of the statistics is the host spot during split-generation, due to the new Text() allocation within String statistics implementation.