[PARQUET-384] Add Dictionary Based Filtering to Filter2 API

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-mr
• Assignee: Daniel Weeks
• Reporter: Daniel Weeks
• Created: 24/Sep/15 22:48
• Updated: 21/Apr/18 12:38
• Resolved: 09/Mar/16 21:22

Description

Dictionary based predicate evaluation show very significant gains for certain cases like the following:

• Sparse values in dictionary encoded columns
• Data staged sorted by fields used in predicate evaluation

This would be good to introduce into the filter2 API.

Issue Links

• blocks: PARQUET-392 Release Parquet-mr 1.9.0
• relates to: PARQUET-548 Add Java metadata for PageEncodingStats
• supercedes: PARQUET-374 Add api to read dictionary from each column chunk for predicate pushdown
• links to: PR #330