[FLINK-13376] FileInputFormat can not be used in batch mode of blink-planner

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: Connectors / FileSystem, Runtime / Task, Table SQL / Runtime
• Labels: pull-request-available
• Assignee: Biao Liu
• Reporter: Jingsong Lee
• Created: 23/Jul/19 05:30
• Updated: 01/Aug/19 10:57
• Resolved: 01/Aug/19 10:14

Description

FileInputFormat use ContinuousFileReaderOperator in StreamExecutionEnvironment, and ContinuousFileReaderOperator join the async thread in close, So before closing, the data is incomplete.

But batch operators use endInput to do some work, this lead to incomplete records in endInput, and lead to wrong results.

This mean batch mode of blink-planner can not use file related sources...

Issue Links

• blocks: FLINK-13436 Add TPC-H queries as E2E tests
• links to: GitHub Pull Request #9221