[FLINK-12149] Support Proto for Streaming File Sink

• Type: New Feature
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Connectors / FileSystem
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
• Assignee: Unassigned
• Reporter: Kailash Hassan Dayanand
• Created: 09/Apr/19 17:35
• Updated: 23/Dec/21 10:39

Description

Currently we are supporting AuroParquetWriters in flink here:https://ci.apache.org/projects/flink/flink-docs-release-1.7/api/java/org/apache/flink/formats/parquet/avro/ParquetAvroWriters.html. Supporting an protoParquetWriter within Flink will be a good addition as well. 

Currently Parquet already have support for ProtoParquetWriters here: https://github.com/apache/parquet-mr/blob/master/parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoParquetWriter.java. We can extend the classes from here and make it available in the appropriate format to support ProtoParquet writing.