[FLINK-18232] Hive streaming source can not work when inserting new multiple records partition with start offset

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Connectors / Hive
• Labels: pull-request-available
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 10/Jun/20 06:24
• Updated: 11/Jun/20 03:16
• Resolved: 11/Jun/20 02:32

Description

• Hive streaming source HiveMapredSplitReader reuse row in a wrong way, if change reuse row instance, will loose partition fields.
• When converting Flink File split to Hadoop File split, length should not be -1.
• DirectoryMonitorDiscovery should convert DFS modificationTime to UTC time mills.
• HiveTableSource.createStreamSourceForNonPartitionTable should use local zone mills instead of UTC mills because ContinuousFileMonitoringFunction use local zone mills.

Issue Links

• relates to: FLINK-18077 E2E tests manually for Hive streaming source
• links to: GitHub Pull Request #12573