Avro
  1. Avro
  2. AVRO-966

Bug in GenericData#resolveUnion when resolving union of null and array

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.1
    • Fix Version/s: 1.6.2
    • Component/s: java
    • Labels:
      None

      Description

      I have a simple avro schema from which I generate an avro specific object:
      {{
      {"type": "record",
      "name": "org.company.Test",
      "fields": [
      { "name": "arr","type": ["null",

      {"type": "array","items": "float" }

      ], "default": null }
      ]
      }
      }}
      Then a simple piece of code to reproduce a bug:
      {{
      Test test = new Test();
      List<Float> list = new ArrayList<Float>();
      list.add(1.1f);
      list.add(2.2f);
      test.setArr(list);

      DataFileWriter<Test> myWriter = new DataFileWriter<Test>(new ReflectDatumWriter(test.getSchema()));
      File f = new File("/tmp/test.avro");
      myWriter.create(test.getSchema(), f);
      myWriter.append(test);
      myWriter.close();
      }}

      I get an exception:
      {{
      Exception in thread "main" org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.UnresolvedUnionException: Not in union ["null",

      {"type":"array","items":"float"}

      ]: [1.1, 2.2]
      at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:261)
      <my code>
      Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["null",

      {"type":"array","items":"float"}

      ]: [1.1, 2.2]
      at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:549)
      at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70)
      at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:102)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:102)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
      at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:255)
      }}

      My investigation showed that in GenericData#resolveUnion method getSchemaName() is called. And the latter method when checks whether datum is a record, succeeds. Why it happens boils down to the fact that in ReflectData#createSchema an "if"-body under case (type instanceof ParameterizedType) is not executed.

      I can supply more details if needed. Or explain in a clear way if I didn't manage to.

      1. AVRO-966-2.patch
        2 kB
        Vyacheslav Zholudev
      2. AVRO-966.patch
        1 kB
        Doug Cutting
      3. AVRO-966.patch
        2 kB
        Doug Cutting

        Activity

          People

          • Assignee:
            Doug Cutting
            Reporter:
            Vyacheslav Zholudev
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development