[FLINK-10272] using Avro's DateConversion causes ClassNotFoundException on Hadoop bundle

• Type: Improvement
• Status: Reopened
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.5.3, 1.6.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Assignee: Unassigned
• Reporter: Roel Van der Paal
• Created: 31/Aug/18 11:19
• Updated: 29/Jul/21 07:37

Description

When using org.apache.avro.data.TimeConversions.DateConversion()
in a job on a Hadoop bundled Flink cluster, it throws a ClassNotFoundException on org.joda.time.ReadablePartial

• it only occurs on the Hadoop bundled Flink cluster, not on the one without Hadoop
• it occurs for both version 1.5.3 and 1.6.0 (I did not check the other versions)
• this is probably because org.apache.avro:avro is included in the flink-shaded-hadoop2-uber-x.x.x.jar, but joda-time not (joda-time is an optional dependency from org.apache.avro:avro)
• adding joda-time to the flink lib folder fixes the problem

Proposed solution is to add joda-time to the flink-shaded-hadoop2-uber-x.x.x.jar or remove org.apache.avro:avro from the flink-shaded-hadoop2-uber-x.x.x.jar.