[AVRO-2400] Avro 1.9.0 can't resolve schemas that can be resolved in 1.8.2

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.9.1
• Component/s: java
• Assignee: Jacob Tolar
• Reporter: Jacob Tolar
• Created: 17/May/19 23:14
• Updated: 04/Feb/22 09:33
• Resolved: 14/Jun/19 08:00

Description

The failure occurs in ResolvingGrammarGenerator when reader and writer schema have an array of records with different full names (e.g. different namespace). 

Exception in thread "main" java.lang.ClassCastException: org.apache.avro.Resolver$ReaderUnion cannot be cast to org.apache.avro.Resolver$Container

Avro 1.8.2 allowed this behavior but it now fails in 1.9.0. Looking at the jiras and code, I don't believe this was intentional ( AVRO-2275,  https://github.com/apache/avro/commit/39d959e1c6a1f339f03dab18289e47f27c10be7f  ).

 

It looks like there were some attempts to keep compatibility ( https://github.com/apache/avro/blob/branch-1.9/lang/java/avro/src/main/java/org/apache/avro/Resolver.java , e.g. see the commented out check for w.getFullName() in resolve()) but this case was missed.

 

See this simple example to reproduce. 

https://gist.github.com/jacobtolar/c88d43ab4e8767227891e5cdc188ffad

Issue Links

• is duplicated by: AVRO-2485 ClassCastException on Nullable Map Value Schemas with Backward Compatible Changes (Field Addition)
• is duplicated by: AVRO-2415 Regression: Schema evolution of union containing array fails in 1.9.0
• is related to: SPARK-27733 Upgrade to Avro 1.10.1
• is related to: AVRO-3133 EnumAdjust.resolve should compare unqualified name rather than full name
• links to: GitHub Pull Request #526