Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9018

Kafka Connect - throw clearer exceptions on serialisation errors

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      When Connect fails on a deserialisation error, it doesn't show if that's the key or value that's thrown the error, nor does it give the user any indication of the topic/partition/offset of the message. Kafka Connect should be improved to return this information.

      Example message that user will get (in this case caused by reading non-Avro data with the Avro converter)

      Caused by: org.apache.kafka.connect.errors.DataException: Failed to deserialize data for topic sample_topic to Avro:
       at io.confluent.connect.avro.AvroConverter.toConnectData(AvroConverter.java:110)
       at org.apache.kafka.connect.runtime.WorkerSinkTask.lambda$convertAndTransformRecord$1(WorkerSinkTask.java:487)
       at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
       at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
       ... 13 more
       Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
       Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mmolimar Mario Molina
                Reporter:
                rmoff Robin Moffatt
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: