[FLINK-1722] Streaming not respecting FinalizeOnMaster for output formats

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: API / DataStream
• Labels: pull-request-available
• Assignee: Haibo Sun
• Reporter: Robert Metzger
• Created: 18/Mar/15 10:45
• Updated: 17/Jun/19 11:18
• Resolved: 17/Jun/19 11:18

Description

The Hadoop output formats execute a process in the end to move the produced files from a temp directory to the final location.

The batch API is annotating output formats that execute something in the end with the FinalizeOnMaster interface.
The streaming component is not respecting this interface. Hence, HadoopOutputFormats aren't writing their final data into the desired destination.

Issue Links

• is required by: FLINK-12568 Implement OutputFormat to write Hive tables
• links to: GitHub Pull Request #8642