[FLINK-27593] Parquet/Orc format reader optimization

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 1.20.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Runtime
• Assignee: Unassigned
• Reporter: dalongliu
• Created: 12/May/22 11:58
• Updated: 11/Mar/24 12:43

Issue Links

• is related to: FLINK-17778 [umbrella] FileSystemTableFactory improve

Sub-Tasks

• Cleanup from Parquet and Orc the partition key handling logic: Open
• ParquetColumnarRowSplitReader#reachedEnd() returns false after it returns true: Open
• Upgrade ORC to 1.5.10 version: Open
• Parquet reader support nested type in array/map type: Open
• Aggregate (Min/Max/Count) push down for ORC: Open
• Aggregate (Min/Max/Count) push down for Parquet: Open
• Push down filter by partition column for Aggregate (Min/Max/Count) for Parquet and Orc: Open
• Push down group by partition column for Aggregate (Min/Max/Count) for Parquet and Orc: Open