[FLINK-19798] Support creating hive table input splits in parallel

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Table SQL / Ecosystem
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Dan Zou
• Created: 25/Oct/20 09:59
• Updated: 10/Nov/21 10:43

Description

For now, we could only creating hive table input splits serially.  
HiveTableInputFormat::createInputSplits cost 40s~60s for a table with 1500 partition in my tests. IMO, we could support creating splits in parallel, which will save most of the time to do this.