Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2471

Java maven plugin code generation doesn't add conversion for timestamp-micros

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.9.0
    • Fix Version/s: None
    • Component/s: java
    • Labels:
      None

      Description

      Field in schema: (there is no single field with timestamp-millis logical type)

      {
        "name": "RECORDING_TIME",
        "type": [
          "null",
          {
            "type": "long",
            "logicalType": "timestamp-micros"
          }
        ],
        "default": null
      }
      

      Maven plugin configuration:

      <plugin>
      	<groupId>org.apache.avro</groupId>
      	<artifactId>avro-maven-plugin</artifactId>
      	<version>1.9.0</version>
      	<executions>
      		<execution>
      			<goals>
      				<goal>schema</goal>
      			</goals>
      			<configuration>
      				<stringType>String</stringType>
      				<enableDecimalLogicalType>true</enableDecimalLogicalType>
      				<sourceDirectory>${project.basedir}/src/main/resources/</sourceDirectory>
      			</configuration>
      		</execution>
      	</executions>
      </plugin>
      

      Part of the generated class:

        private static SpecificData MODEL$ = new SpecificData();
      static {
          MODEL$.addLogicalTypeConversion(new org.apache.avro.data.TimeConversions.DateConversion());
          MODEL$.addLogicalTypeConversion(new org.apache.avro.data.TimeConversions.TimestampMillisConversion()); // <--- this should be TimestampMicrosConversion
          MODEL$.addLogicalTypeConversion(new org.apache.avro.Conversions.DecimalConversion());
        }
      

      For example this code:

      Data data = Data.newBuilder()
      		.setRECORDINGTIME(Instant.now())
                      .build();
      

      Fails during comparison:

      org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
      
      Caused by: org.apache.avro.AvroRuntimeException: Unknown datum type java.time.Instant: 2019-07-12T14:24:47.322Z
      	at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:887)
      	at org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:420)
      	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:850)
      	at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:249)
      	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:142)
      	at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:98)
      	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:195)
      	at org.apache.avro.specific.SpecificDatumWriter.writeRecord(SpecificDatumWriter.java:83)
      	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:130)
      	at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:98)
      	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:195)
      	at org.apache.avro.specific.SpecificDatumWriter.writeRecord(SpecificDatumWriter.java:83)
      	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:130)
      	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)
      	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:92)
      	at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53)
      	at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:65)
      	at org.apache.kafka.common.serialization.ExtendedSerializer$Wrapper.serialize(ExtendedSerializer.java:55)
      	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:841)
      	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803)
      	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:690)
      

      When manually changed to org.apache.avro.data.TimeConversions.TimestampMicrosConversion everything works properly.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rskraba Ryan Skraba
                Reporter:
                mtracz Marek Tracz
              • Votes:
                4 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: