Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-140

Allow clients to control the GenericData object that is used to read Avro records

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      Right now, Parquet always uses the default SpecificData instance (retrieved by SpecificData.get()) to lookup the schemas for SpecificRecord subclasses. Unfortunately, if the definition of the SpecificRecord subclass is not available to the classloader used in SpecificData.get(), we will fail to find the definition of the SpecificRecord subclass and will fall back to returning a GenericRecord, which will cause a ClassCastException in any client code that is expecting an instance of the SpecificRecord subclass.

      We can fix this limitation by allowing the client code to specify how to construct a custom instance of SpecificData (or any other subclass of GenericData) for Parquet to use, including instances of SpecificData that use alternative classloaders.

        Attachments

          Activity

            People

            • Assignee:
              jwills Josh Wills
              Reporter:
              jwills Josh Wills
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: