[PARQUET-140] Allow clients to control the GenericData object that is used to read Avro records

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: parquet-mr
• Assignee: Josh Wills Assign to me
• Reporter: Josh Wills
• Created: 01/Dec/14 22:17
• Updated: 28/Oct/15 16:39
• Resolved: 02/Dec/14 16:22

Description

Right now, Parquet always uses the default SpecificData instance (retrieved by SpecificData.get()) to lookup the schemas for SpecificRecord subclasses. Unfortunately, if the definition of the SpecificRecord subclass is not available to the classloader used in SpecificData.get(), we will fail to find the definition of the SpecificRecord subclass and will fall back to returning a GenericRecord, which will cause a ClassCastException in any client code that is expecting an instance of the SpecificRecord subclass.

We can fix this limitation by allowing the client code to specify how to construct a custom instance of SpecificData (or any other subclass of GenericData) for Parquet to use, including instances of SpecificData that use alternative classloaders.