[AVRO-2380] Logical types are not supported when the actual datatype is also present in Union

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.8.2
• Assignee: Unassigned
• Reporter: Harshvardhan Agrawal
• Created: 25/Apr/19 22:12
• Updated: 27/Nov/20 12:57

Description

{
"name" : "data",
"type" : {
 "type" : "map",
 "values" : {
  "type" : "array",
  "items" : {
   "type" : "map",
   "values" : [ "string", "long", "null", "double", "float", "int", "boolean" ]
  }
 }
}

The above schema represents a Map<String, List<Map<String, Object>>> where the values of the inner map could be any of the basic datatypes supported by Avro.

If we want the Map to support logical types such as timestamp for e.g.

{
"name" : "data",
"type" : {
 "type" : "map",
 "values" : {
  "type" : "array",
  "items" : {
   "type" : "map",
   "values" : [ "string", "long", "null", "double", "float", "int", "boolean",
                 { "type" : "long", "logicalType" : "timestamp-micros" }]
  }
 }
}

The schema parser fails with the following error:

 

Exception in thread "main" org.apache.avro.AvroRuntimeException: Duplicate in union:long at org.apache.avro.Schema$UnionSchema.<init>(Schema.java:854)
at org.apache.avro.Schema.parse(Schema.java:1341)
at org.apache.avro.Schema.parse(Schema.java:1311)
at org.apache.avro.Schema.parse(Schema.java:1306)
at org.apache.avro.Schema.parse(Schema.java:1311)
at org.apache.avro.Schema.parse(Schema.java:1269)
at org.apache.avro.Schema$Parser.parse(Schema.java:1032)
at org.apache.avro.Schema$Parser.parse(Schema.java:1020)