[AVRO-1341] Allow controlling avro via java annotations when using reflection.

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.5
• Component/s: java

Release Note

Added the java annotations @AvroName, @AvroIgnore, @AvroMeta, @AvroAlias and @AvroEncode to control the behavior of avro when using reflection.

• Assignee: Vincenz Priesnitz
• Reporter: Vincenz Priesnitz
• Created: 28/May/13 13:52
• Updated: 28/Nov/17 18:09
• Resolved: 07/Aug/13 23:17

Description

It would be great if one could control avro with java annotations. As of now, it is already possible to mark fields as Nullable or classes being encoded as a String. I propose a bigger set of annotations to control the behavior of avro on fields and classes. Such annotations have proven useful with jacksons json serialization and morphias mongoDB serialization.

I propose the following additional annotations:
@AvroName("alternativeName")
@AvroAlias(alias="alias", space="space")
@AvroIgnore
@AvroMeta(key="K", value="V")
@AvroEncode(using=CustomEncoding.class)

Java fields with the @AvroName("alternativeName") annotation will be renamed in the induced schema. When reading an avro file via reflection, the reflection reader will look for fields in the schema with "alternativeName".
For example:

   @AvroName("foo")
   int bar;  

is serialized as

  { "name" : "foo", "type" : "int" } 

The @AvroAlias annotation will add a new alias to the induced schema of a record, enum or field. The space parameter is optional and defaults to the namespace of the named schema the alias is added to.

Fields with the @AvroIgnore annotation will be treated as if they had a transient modifier, i.e. they will not be written to or read from avro files.

The @AvroMeta(key="K", value="V") annotation allows you to store an arbitrary key : value pair at every node in the schema.

   @AvroMeta(key="fieldKey", value="fieldValue")
   int foo;  

will create the following schema

{"name" : "foo", "type" : "int", "fieldKey" : "fieldValue" } 

Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class) annotation. This annotation is a generalization of the @Stringable annotation. The @Stringable annotation is limited to classes with string argument constructors. Some classes can be similarly reduced to a smaller class or even a single primitive, but dont fit the requirements for @Stringable. A prominent example is java.util.Date, which instances can essentially be described with a single long. Such classes can now be encoded with a CustomEncoding, which reads and writes directly from the encoder/decoder.

One simply extends the abstract CustomEncodings class by implementing a schema, a read method and a write method. A java field can then be annotated like this:

@AvroEncode(using=DateAslongEncoding.class)
Date date;

The custom encoding implementation would look like

public class DateAsLongEncoding extends CustomEncoding<Date> {
  {
    schema = Schema.create(Schema.Type.LONG);
    schema.addProp("CustomEncoding", "DateAsLongEncoding");
  }
  
  @Override
  public void write(Object datum, Encoder out) throws IOException {
    out.writeLong(((Date)datum).getTime());
  }
  
  @Override
  public Date read(Object reuse, Decoder in) throws IOException {
    if (reuse != null) {
      ((Date)reuse).setTime(in.readLong());
      return (Date)reuse;
    }
    else return new Date(in.readLong());
  }
}

I implemented said annotations and a custom encoding for java.util.Date as a proof of concept and also extended the @Stringable annotations to fields.

This issue is a followup of AVRO-1328 and AVRO-1330.

Issue Links

• contains: AVRO-1328 Java: Additional annotations for reflection
• contains: AVRO-1330 java: allow custom Encodings with annotations
• is related to: AVRO-2108 @Stringable annotation should be processed before creating the schema
• is related to: AVRO-1498 Custom Encoding with @AvroEncode does not work when FieldAccessReflect is used
• is related to: AVRO-739 Add Date/Time data types
• relates to: AVRO-1658 Add avroDoc on reflect
• supercedes: AVRO-1328 Java: Additional annotations for reflection
• supercedes: AVRO-1330 java: allow custom Encodings with annotations