[AVRO-3156] Performance degradation in SpecificRecordBuilder introduced in 1.9.0

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.9.0, 1.10.0
• Component/s: java
• Labels: pull-request-available

Environment

Using SpecificData in environments with multiple classloaders.

• Assignee: Unassigned
• Reporter: Steven Aerts
• Created: 11/Jun/21 12:27
• Updated: 04/Mar/22 10:25
• Resolved: 04/Mar/22 10:23

Description

The change introduced in spark 1.9.0 which changed:

 SpecificData.get()

into:

SpecificData.getForSchema(schema)

introduced a significant performance degradation in environments where the class of schema is provided by a different classloader then the classloader containing SpecificData.

A possible solution is to use the classCache of the default SpecificData so the sometimes expensive classloader codepath is cached. (PR coming up)

We noticed this in after trying out a spark upstep from spark 3.1.0 (avro 1.8.2) to 3.2.0 (spark 1.10.2) where 74% of the time was spend in millions of times resolving the same class.

With this patch this resolving time was brought back from 74% to 0.70%.

JMC flamegraph showing this issue:

 

Issue Links

• blocks: SPARK-35744 Performance degradation in avro SpecificRecordBuilders
• duplicates: AVRO-3186 Java: Avro can't decode union<null, logicalType> field of a record
• is related to: AVRO-3048 Using builders leads to performance degradation
• links to: GitHub Pull Request #1253