Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-797

Built-in support for "any" types

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: java, spec
    • Labels:
      None

      Description

      In my application, I generate an Avro protocol programmatically based on a model. While my model permits me to include things of type java.util.Object or java.lang.Enum, Avro only supports these notions via the concept of unions.

      Therefore, to support this functionality, I iterate over my list of schemas and create a union of every type (or every enum type) in the protocol (and the null schema) to create my "any type" schema. This would be sufficient except that this new schema doesn't exist when I create the schemas in my model that would use it, so I need to put in a bogus placeholder schema, and this creates the added complexity of needing to programatically reconstruct my schemas, which is risky and problematic at best.

      What I propose to address this is a new type of schema that acts as a layer of indirection and is not resolved during protocol generation (so no schema reconstruction is necessary).

      Use Case #1: Translate generic "object" from model into Avro Schema

      Field field = new Field("payload", Schema.createAny(), null, null); // default - any named type
      List<Field> fields = new ArrayList<Field>();
      fields.add(field);
      Schema record = Schema.createRecord("Foo", null, "com.acme", false);
      record.setFields(fields);

      Use Case #2: Translate generic "enum" from model into Avro Schema

      Field field = new Field("payload", Schema.createAny(Type.ENUM), null, null); // specifies a placeholder only for enums
      List<Field> fields = new ArrayList<Field>();
      fields.add(field);
      Schema record = Schema.createRecord("Foo", null, "com.acme", false);
      record.setFields(fields);

      The consumer of the protocol should treat an "any" type schema as the union of every named type in the protocol, constrained by the specified Schema type.

      If there is sufficient interest, I would be willing to provide a Java patch for this functionality.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dkaminsky Douglas Kaminsky
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: