[IMPALA-4989] Improve filtering based on parquet::Statistics

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.12.0
• Component/s: Backend, Frontend
• Labels: parquet
• Assignee: Lars Volker
• Reporter: Lars Volker
• Created: 24/Feb/17 22:32
• Updated: 07/May/18 16:06
• Resolved: 07/May/18 16:06

Description

This is an umbrella JIRA to track various related issues around writing and reading parquet::Statistics

Issue Links

• is a parent of: IMPALA-4977 Evaluate IN predicates against parquet::Statistics
• is a parent of: IMPALA-4982 Add a test for statistics based filtering of row groups for root-level scalar columns of parquet files with nested types
• is a parent of: IMPALA-4815 Populate min/max statistics in Parquet files for Decimal types
• is a parent of: IMPALA-4817 Populate min/max statistics in Parquet files for String values
• is a parent of: IMPALA-4819 Populate min/max statistics in Parquet files for Timestamp values
• is a parent of: IMPALA-4985 Evaluate parquet::Statistics to skip data in nested types
• is a parent of: IMPALA-4988 Add query option to control filtering based on parquet::Statistics
• is a parent of: IMPALA-5061 Populate null_count in parquet::statistics in the parquet table writer
• is a parent of: IMPALA-5062 Populate distinct_count in parquet::statistics in the parquet table writer
• is a parent of: IMPALA-6113 Skip row groups with predicates on NULL columns
• is a parent of: IMPALA-5185 Parquet scanner should skip pages based on parquet page statistics
• is related to: IMPALA-5009 Clean up test_insert_parquet.py
• relates to: IMPALA-3430 Runtime filter : Extend runtime filter to support Min/Max values for HDFS scans
• relates to: IMPALA-4958 Simplify binary predicates in the FE