Avro
  1. Avro
  2. AVRO-873

SpecificDatumReader should allow users to specify the ClassLoader used.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: java
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Right now SpecificData resolves classes for SpecificRecords by calling Class.forName(), which uses the ClassLoader used to load the SpecificData class. I believe this makes it impossible to deserialize into SpecificRecords that are loaded using a different ClassLoader.

      One relatively simple way to fix this limitations involves:

      • Adding a constructor to SpecificData that allows the user to specify a ClassLoader.
      • Changing the current protected constructor of SpecificDatumReader that takes a SpecificData to be public instead of protected.
      • Changing SpecificDatumReader to actually use the SpecificData that is passed into the constructor.

        Issue Links

          Activity

          Hide
          Doug Cutting added a comment -

          I committed this. Thanks, Michael.

          I made a few minor changes, javadoc mostly...

          Show
          Doug Cutting added a comment - I committed this. Thanks, Michael. I made a few minor changes, javadoc mostly...
          Hide
          Michael Armbrust added a comment -

          Thanks for the feedback. Here is an update patch.

          Show
          Michael Armbrust added a comment - Thanks for the feedback. Here is an update patch.
          Hide
          Doug Cutting added a comment -

          Looks like a useful addition.

          Perhaps instead of adding a new field to SpecificDatumReader we can add a new method to SpecificData, since the base GenericData already has a field containing the desired value:

          public SpecificData getSpecificData() { return (SpecificData)getData(); }
          

          Then this can be used in SpecificDatumReader, as getSpecificData().getClass(...).

          Also, I'd prefer if the classLoader field were defined nearer the top of the class, since it's used by both a constructor and the getClass() implementation. I'd place it just before the constructors.

          Show
          Doug Cutting added a comment - Looks like a useful addition. Perhaps instead of adding a new field to SpecificDatumReader we can add a new method to SpecificData, since the base GenericData already has a field containing the desired value: public SpecificData getSpecificData() { return (SpecificData)getData(); } Then this can be used in SpecificDatumReader, as getSpecificData().getClass(...). Also, I'd prefer if the classLoader field were defined nearer the top of the class, since it's used by both a constructor and the getClass() implementation. I'd place it just before the constructors.
          Hide
          Michael Armbrust added a comment -

          Here is a patch the implements the improvement. Comments or suggestions appreciated.

          Show
          Michael Armbrust added a comment - Here is a patch the implements the improvement. Comments or suggestions appreciated.

            People

            • Assignee:
              Michael Armbrust
              Reporter:
              Michael Armbrust
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development