[FLINK-33058] Support for JSON-encoded Avro

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Labels: avro flink flink-formats pull-request-available
• Assignee: Dale Lane
• Reporter: Dale Lane
• Created: 07/Sep/23 09:36
• Updated: 21/Nov/23 21:29

Description

Avro supports two serialization encoding methods: binary and JSON

cf. https://avro.apache.org/docs/1.11.1/specification/#encodings 

flink-avro currently has a hard-coded assumption that Avro data is binary-encoded (and cannot process Avro data that has been JSON-encoded).

I propose adding a new optional format option to flink-avro: avro.encoding

It will support two options: 'binary' and 'json'. 

It unset, it will default to 'binary' to maintain compatibility/consistency with current behaviour. 

Issue Links

• links to: @dev discussion
• links to: GitHub Pull Request #23395