[IMPALA-3900] Add per-split runtime filtering to HdfsParquetScanner::ProcessSplit()

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Labels: runtime-filters
• Target Version: Product Backlog
• Assignee: Henry Robinson
• Reporter: Henry Robinson
• Created: 22/Jul/16 22:09
• Updated: 19/Jun/20 19:42
• Resolved: 19/Jun/20 19:42

Description

If a partition filter arrives after a footer scan range for a Parquet has been issued, but before HdfsParquetScanner::ProcessSplit(), there's an opportunity to filter out all the scan ranges that would otherwise be issued when reading that footer, by adding a call to ScanRangeIsFilteredOut() at the top of that method.

Care must be taken to ensure that all scan ranges are marked as done, since they won't be processed by their own scanner instances. This will avoid a recurrence of IMPALA-3804.

Issue Links

• duplicates: IMPALA-3804 Re-enable per-scan filtering for sequence-based scanners