Avro
  1. Avro
  2. AVRO-1341

Allow controlling avro via java annotations when using reflection.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.5
    • Component/s: java
    • Labels:
      None
    • Release Note:
      Hide
      Added the java annotations @AvroName, @AvroIgnore, @AvroMeta, @AvroAlias and @AvroEncode to control the behavior of avro when using reflection.

      Show
      Added the java annotations @AvroName, @AvroIgnore, @AvroMeta, @AvroAlias and @AvroEncode to control the behavior of avro when using reflection.

      Description

      It would be great if one could control avro with java annotations. As of now, it is already possible to mark fields as Nullable or classes being encoded as a String. I propose a bigger set of annotations to control the behavior of avro on fields and classes. Such annotations have proven useful with jacksons json serialization and morphias mongoDB serialization.

      I propose the following additional annotations:
      @AvroName("alternativeName")
      @AvroAlias(alias="alias", space="space")
      @AvroIgnore
      @AvroMeta(key="K", value="V")
      @AvroEncode(using=CustomEncoding.class)

      Java fields with the @AvroName("alternativeName") annotation will be renamed in the induced schema. When reading an avro file via reflection, the reflection reader will look for fields in the schema with "alternativeName".
      For example:

         @AvroName("foo")
         int bar;  
      

      is serialized as

        { "name" : "foo", "type" : "int" } 
      

      The @AvroAlias annotation will add a new alias to the induced schema of a record, enum or field. The space parameter is optional and defaults to the namespace of the named schema the alias is added to.

      Fields with the @AvroIgnore annotation will be treated as if they had a transient modifier, i.e. they will not be written to or read from avro files.

      The @AvroMeta(key="K", value="V") annotation allows you to store an arbitrary key : value pair at every node in the schema.

         @AvroMeta(key="fieldKey", value="fieldValue")
         int foo;  
      

      will create the following schema

      {"name" : "foo", "type" : "int", "fieldKey" : "fieldValue" } 
      

      Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class) annotation. This annotation is a generalization of the @Stringable annotation. The @Stringable annotation is limited to classes with string argument constructors. Some classes can be similarly reduced to a smaller class or even a single primitive, but dont fit the requirements for @Stringable. A prominent example is java.util.Date, which instances can essentially be described with a single long. Such classes can now be encoded with a CustomEncoding, which reads and writes directly from the encoder/decoder.

      One simply extends the abstract CustomEncodings class by implementing a schema, a read method and a write method. A java field can then be annotated like this:

      @AvroEncode(using=DateAslongEncoding.class)
      Date date;
      

      The custom encoding implementation would look like

      public class DateAsLongEncoding extends CustomEncoding<Date> {
        {
          schema = Schema.create(Schema.Type.LONG);
          schema.addProp("CustomEncoding", "DateAsLongEncoding");
        }
        
        @Override
        public void write(Object datum, Encoder out) throws IOException {
          out.writeLong(((Date)datum).getTime());
        }
        
        @Override
        public Date read(Object reuse, Decoder in) throws IOException {
          if (reuse != null) {
            ((Date)reuse).setTime(in.readLong());
            return (Date)reuse;
          }
          else return new Date(in.readLong());
        }
      }
      

      I implemented said annotations and a custom encoding for java.util.Date as a proof of concept and also extended the @Stringable annotations to fields.

      This issue is a followup of AVRO-1328 and AVRO-1330.

      1. AVRO-1341.patch
        32 kB
        Vincenz Priesnitz
      2. AVRO-1341.patch
        32 kB
        Vincenz Priesnitz
      3. AVRO-1341.patch
        26 kB
        Vincenz Priesnitz
      4. AVRO-1341.patch
        38 kB
        Vincenz Priesnitz
      5. AVRO-1341.patch
        39 kB
        Doug Cutting
      6. AVRO-1341.patch
        39 kB
        Vincenz Priesnitz
      7. AVRO-1341.patch
        40 kB
        Doug Cutting

        Issue Links

          Activity

            People

            • Assignee:
              Vincenz Priesnitz
              Reporter:
              Vincenz Priesnitz
            • Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development