[FLINK-11395] Support for Avro StreamingFileSink

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.11.0
• Component/s: Connectors / FileSystem
• Labels: pull-request-available usability
• Assignee: Yun Gao
• Reporter: Elango Ganesan
• Created: 18/Jan/19 19:50
• Updated: 15/May/20 12:40
• Resolved: 15/May/20 12:40

Description

Current implementation for StreamingFileSink supports Rowformat for text and json files . BulkWriter has support for Parquet files . Out of the box RowFormat does not seem to be right fit for writing Avro files as avro files need to persist metadata when opening new file.  We are thinking of implementing Avro writers similar to how Parquet is implemented . I am happy to submit a PR if this approach sounds good . 

Issue Links

• links to: GitHub Pull Request #11385