[FLINK-9750] Create new StreamingFileSink that works on Flink's FileSystem abstraction

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: Connectors / Common
• Labels: pull-request-available
• Assignee: Kostas Kloudas
• Reporter: Stephan Ewen
• Created: 04/Jul/18 20:11
• Updated: 29/Jul/18 18:55
• Resolved: 19/Jul/18 09:59

Description

Using Flink's own file system abstraction means that we can add additional streaming/checkpointing related behavior.

In addition, the new StreamingFileSink should only rely on internal checkpointed state what files are possibly in progress or need to roll over, never assume enumeration of files in the file system.

Issue Links

• links to: GitHub Pull Request #6281
• links to: GitHub Pull Request #6281