[FLINK-18192] Upgrade to Avro version 1.10.0 from 1.8.2

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Labels: pull-request-available

Release Note

The default version of avro in flink-avro module was upgraded to 1.10. If for some reason you need an older version (you have avro coming from hadoop, or you use classes generated from an older avro version), please explicitly downgrade the avro version in your project.

NOTE: We observed a decreased performance of the Avro 1.10 version compared to 1.8.2. If you are concerned with the performance and you are fine working with an older version of Avro, consider downgrading the Avro version.

• Assignee: Dawid Wysakowicz
• Reporter: Lucas Heimberg
• Created: 08/Jun/20 18:59
• Updated: 03/Nov/20 14:18
• Resolved: 14/Sep/20 19:52

Description

As of version 1.11, Flink (i.e., flink-avro) still uses Avro in version 1.8.2.

Avro 1.9.2 contains many bugfixes, in particular in respect to the support for logical types. A further advantage would be that an upgrade to Avro 1.9.2 would also allow to use the Confluent Schema Registry client and Avro deserializer in version 5.5.0, which finally support schema references.

Therefore it would be great if Flink could make use of Avro 1.9.2 or higher in future releases.

 

 

Issue Links

• causes: FLINK-19440 Performance regression on 15.09.2020
• is cloned by: FLINK-18546 Upgrade to Kafka Schema Registry Client 5.5.0
• relates to: FLINK-18546 Upgrade to Kafka Schema Registry Client 5.5.0
• relates to: FLINK-18802 Create an uber jar when packaging flink-avro for sql client
• supercedes: FLINK-12532 Upgrade Avro to version 1.10.0
• links to: GitHub Pull Request #13903