[HUDI-5068] Support cow flink batch create fs input split asynchronously

• Type: Improvement
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Component/s: flink-sql, incremental-query, performance
• Labels: pull-request-available
• Assignee: Shizhi Chen
• Reporter: Shizhi Chen
• Created: 21/Oct/22 10:44
• Updated: 10/Mar/23 01:50

Description

CopyOnWriteInputFormat#createInputSplits is invoked by org.apache.flink.runtime.executiongraph.ExecutionJobVertex in JobManager to create file input splits synchronously.

It's found that in batch mode this will occupy the largest share of job submission time.

So in this PR it will be optimized by creating input splits in thread pool executor asynchronously.

Issue Links

• links to: GitHub Pull Request #7019