Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1865

GenericData.Array class missing no arg constructor for Kryo serialization

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 1.8.1
    • None
    • java

    Description

      When trying to serialize Java classes generated by Avro that contain a field as follows:
      array<MyClass> myclasses = []

      serialization fails with the following error message
      com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor)

      Minimum example to recreate this:
      Avro IDL definition:

      @namespace("com.adello")
      protocol KryoTest {
      	record TestRecord {
      		array<int> values = [];
      	}
      }
      

      This gets compiled using java -jar avro-tools-1.8.1.jar idl2schemata kryo.avdl .
      Resulting Avro schema:

      {
        "type" : "record",
        "name" : "TestRecord",
        "namespace" : "com.adello",
        "fields" : [ {
          "name" : "values",
          "type" : {
            "type" : "array",
            "items" : "int"
          },
          "default" : [ ]
        } ]
      }
      

      Generate Java class using java -jar avro-tools-1.8.1.jar compile schema TestRecord.avsc

      Kryo serialization test code:

      Kryo kryo = new Kryo();
      try {
                  Output output = new Output(new FileOutputStream("test.bin"));
                  TestRecord o = TestRecord.newBuilder().build();
                  System.out.println("Before serialization: ");
                  System.out.println(o.toString());
                  kryo.writeObject(output, o);
                  output.close();
      
                  Input input = new Input(new FileInputStream("test.bin"));
                  TestRecord i = kryo.readObject(input, TestRecord.class);
                  input.close();
                  System.out.println("After deserialization: ");
                  System.out.println(i.toString());
      
                  System.out.println(Objects.deepEquals(o, i));
              } catch (FileNotFoundException e) {
                  e.printStackTrace();
              }
      

      This is a common enough issue with Kryo that they have a dedicated section on their website on it: http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception

      The attached patch adds a no args constructor for the inner class Array of GenericData with zero entries and a dummy schema. My personal tests for serialization with Kryo were successful when using it. Since I do not have complete insight into Avro I'd like to know if this could be a breaking change and how to test it if so.

      Attachments

        1. GenericArray-no-args-ctor-kryo.patch
          1 kB
          Alexander Kasper

        Activity

          People

            AnLei Alexander Kasper
            AnLei Alexander Kasper
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: