[FLINK-14799] Improve batch sql and hive integrate performance milestone-2

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Connectors / Hive, Table SQL / Planner
• Assignee: Unassigned
• Reporter: Jingsong Lee
• Created: 15/Nov/19 03:29
• Updated: 02/Mar/20 04:23

Issue Links

• is related to: FLINK-14133 Improve batch sql and hive integrate performance milestone-1

Sub-Tasks

• Introduce parallelism inference for source: Closed
• Introduce parquet ColumnarRow split reader: Resolved
• Investigate pushing predicate/projection to underlying Hive input format: Open
• Implement LimitableTableSource for FileSystemTableFactory: Closed
• Multi vectorized read version support for hive orc read: Closed
• Provide a new InputSplitAssigner that consider page cache utilization ratio: Open
• Reusing record object in StreamTaskNetworkInput: Open
• Optimize the operator algorithm to lazily allocate memory: Resolved
• List/Map/Struct types support for vectorized orc reader: Resolved
• Introduce WritableVectors for abstract writing: Resolved
• Integrate parquet columnar row reader to hive: Resolved
• Provide an assigner strategy of average splits allocation: Open