[FLINK-18202] Introduce Protobuf format

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.16.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
• Labels: auto-unassigned pull-request-available sprint stale-assigned
• Release Note: Apache Flink now support the Protocol Buffers (Protobuf) format. This allows you to use this format directly in your Table API or SQL applications.
• Sprint: Flink SQL Sprint
• Assignee: Suhan Mao
• Reporter: Benchao Li
• Created: 09/Jun/20 07:02
• Updated: 01/Aug/22 11:39
• Resolved: 27/Jul/22 14:51

Description

PB[1] is a very famous and wildly used (de)serialization framework. The ML[2] also has some discussions about this. It's a useful feature.

This issue maybe needs some designs, or a FLIP.

[1] https://developers.google.com/protocol-buffers

[2] http://apache-flink.147419.n8.nabble.com/Flink-SQL-UDF-td3725.html

Issue Links

• blocks: FLINK-20421 Support canal-protobuf format
• duplicates: FLINK-20179 Support Protobuf serialization and deseriazation schema for RowData type
• is related to: FLINK-28721 Support Protobuf in DataStream API
• is related to: FLINK-9604 Support KafkaProtoBufTableSource
• relates to: FLINK-28765 Create a doc for protobuf format
• links to: GitHub Pull Request #14376