[FLINK-18181] StreamingFileCommitter should not use fs modification time for proc committer

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Connectors / FileSystem
• Labels: pull-request-available
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 08/Jun/20 10:32
• Updated: 09/Jun/20 16:14
• Resolved: 09/Jun/20 09:14

Description

The filesystem path modification time is the last modification time of all files in directory. 

If there is new data in partition all the time, it will never end.

We should use earlier time for committer.

Issue Links

• relates to: FLINK-18025 E2E tests manually for Hive streaming sink
• links to: GitHub Pull Request #12526