Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-547

AvroType operators can create nested unions

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: IO
    • Labels:
      None

      Description

      When using avro reflection to create PTypes, the type might represent a union type, as specified by the @Union annotation. Derived types from this, such as tableOf() and collections() will create a null union from this (already union) type and create a nested union which throws an AvroRuntimeException.

      @org.apache.avro.reflect.Union(

      { Foo1.class, Foo2.class }

      )
      public static interface Foo {
      public String getFoo();
      }
      public static class Foo1 implements Foo {
      private final Double value = 1.0;
      @Override
      public String getFoo()

      { return value.toString(); }
      }
      public static class Foo2 implements Foo {
      private final Integer value = 2;
      @Override
      public String getFoo() { return value.toString(); }

      }

      Schema schema = ReflectData.get().getSchema(Foo.class);
      AvroType<Foo> ptype = Avros.reflects(Foo.class, schema);

      Avros.tableOf(Avros.strings(), ptype);
      Avros.collections(ptype);

      Exception in thread "main" org.apache.avro.AvroRuntimeException: Nested union: [["double","string"],"null"]
      ...

      Seems like the methods Avros.allowNulls() and AvroTableType.nullable() both are calling Schema.createUnion(unionSchema, nullSchema), and could add an extra case statement to use the already existing union schema.

        Attachments

        1. CRUNCH-547b.patch
          5 kB
          Josh Wills
        2. CRUNCH-547.patch
          4 kB
          Josh Wills

          Activity

            People

            • Assignee:
              jwills Josh Wills
              Reporter:
              aeckstein Adric Eckstein
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: