[HADOOP-12527] Upgrade Avro dependency to 1.7.7 or later

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.7.1
• Component/s: build
• Assignee: Sean Busbey
• Reporter: Jonathan Kelly
• Created: 29/Oct/15 00:51
• Updated: 25/Mar/19 18:15
• Resolved: 12/Apr/18 02:01

Description

Hadoop has depended upon Avro 1.7.4 for a couple of years now (see HADOOP-9672), but Apache Spark depends upon what is currently the latest version of Avro (1.7.7).

This can cause issues if Spark is configured to include the full Hadoop classpath, as the classpath would then contain both Avro 1.7.4 and 1.7.7, with the 1.7.4 classes possibly winning depending on ordering. Here is an example of this issue: http://stackoverflow.com/questions/33159254/avro-error-on-aws-emr/33403111#33403111

Would it be possible to upgrade Hadoop's Avro dependency to 1.7.7 now?

Issue Links

• is depended upon by: HADOOP-9991 Fix up Hadoop POMs, roll up JARs to latest versions
• is duplicated by: HADOOP-14992 Upgrade Avro patch version
• is related to: HBASE-16242 Upgrade Avro to 1.7.7
• is related to: SPARK-16617 Upgrade to Avro 1.8.x
• relates to: HADOOP-11755 Update avro version to have PowerPC supported Snappy-java
• links to: GitHub Pull Request #39
• links to: GitHub Pull Request #39