Avro
  1. Avro
  2. AVRO-1512

Error serializing TUnion with avro-thrift

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.6
    • Fix Version/s: 1.7.7
    • Component/s: java
    • Labels:
      None

      Description

      Attempting to serialize a Thrift union type (TUnion) with avro-thrift produces an error like the following:

      org.apache.avro.AvroRuntimeException: Unknown datum type org.apache.avro.thrift.test.FooOrBar: <FooOrBar foo:foo>
      	at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:639)
      	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604)
      	at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
      	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
      	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
      	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
      	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
      	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
      	at org.apache.avro.thrift.TestThrift.check(TestThrift.java:83)
      	at org.apache.avro.thrift.TestThrift.testStruct(TestThrift.java:60)
      

      Upon investigation it appears that ThriftData#isRecord intentionally does not accept TUnion types as Avro records and so GenericData is unable to determine an Avro type for the datum. However, ThriftData#getSchema does generate an Avro record schema for Thrift union types.

      1. AVRO-1512.patch
        19 kB
        Doug Cutting
      2. test.thrift
        1 kB
        Will Gorman

        Activity

        Will Gorman created issue -
        Will Gorman made changes -
        Field Original Value New Value
        Description Attempting to serialize a Thrift union type (TUnion) with avro-thrift produces an error like the following:

        {code}
        org.apache.avro.AvroRuntimeException: Unknown datum type org.apache.avro.thrift.test.FooOrBar: <FooOrBar foo:foo>
        at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:639)
        at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604)
        at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
        at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
        at org.apache.avro.thrift.TestThrift.check(TestThrift.java:83)
        at org.apache.avro.thrift.TestThrift.testStruct(TestThrift.java:60)
        {code}

        Upon investigation it appears that ThriftData#isRecord intentionally does not accept TUnion types as Avro records and so GenericData is unable to determine an Avro type for the datum. However, ThriftData#getSchema does generate an Avro record schema for Thrift union types.
        Attempting to serialize a Thrift union type (TUnion) with avro-thrift produces an error like the following:

        {noformat}
        org.apache.avro.AvroRuntimeException: Unknown datum type org.apache.avro.thrift.test.FooOrBar: <FooOrBar foo:foo>
        at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:639)
        at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:604)
        at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
        at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
        at org.apache.avro.thrift.TestThrift.check(TestThrift.java:83)
        at org.apache.avro.thrift.TestThrift.testStruct(TestThrift.java:60)
        {noformat}

        Upon investigation it appears that ThriftData#isRecord intentionally does not accept TUnion types as Avro records and so GenericData is unable to determine an Avro type for the datum. However, ThriftData#getSchema does generate an Avro record schema for Thrift union types.
        Will Gorman made changes -
        Attachment test.thrift [ 12645803 ]
        Doug Cutting made changes -
        Attachment AVRO-1512.patch [ 12646633 ]
        Doug Cutting made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Doug Cutting [ cutting ]
        Fix Version/s 1.7.7 [ 12326041 ]
        Doug Cutting made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Will Gorman
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development