[DRILL-5796] Filter pruning for multi rowgroup parquet file

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.15.0
• Component/s: Storage - Parquet
• Labels: ready-to-commit
• Assignee: Jean-Blas IMBERT
• Reporter: Damien Profeta
• Reviewer: Vlad Rozov
• Created: 16/Sep/17 00:59
• Updated: 09/Jan/19 01:49
• Resolved: 01/Aug/18 18:37

Description

Today, filter pruning use the file name as the partitioning key. This means you can remove a partition only if the whole file is for the same partition. With parquet, you can prune the filter if the rowgroup make a partition of your dataset as the unit of work if the rowgroup not the file.

Issue Links

• is related to: DRILL-6857 Limit is not being pushed into scan when selecting from a parquet file with multiple row groups.
• links to: GitHub Pull Request #1298