Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9018

Kafka Connect - throw clearer exceptions on serialisation errors

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.5.0, 2.4.1
    • Fix Version/s: 2.7.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      When Connect fails on a deserialisation error, it doesn't show if that's the key or value that's thrown the error, nor does it give the user any indication of the topic/partition/offset of the message. Kafka Connect should be improved to return this information.

      Example message that user will get (in this case caused by reading non-Avro data with the Avro converter)

      Caused by: org.apache.kafka.connect.errors.DataException: Failed to deserialize data for topic sample_topic to Avro:
       at io.confluent.connect.avro.AvroConverter.toConnectData(AvroConverter.java:110)
       at org.apache.kafka.connect.runtime.WorkerSinkTask.lambda$convertAndTransformRecord$1(WorkerSinkTask.java:487)
       at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
       at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
       ... 13 more
       Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
       Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              mmolimar Mario Molina
              Reporter:
              rmoff Robin Moffatt

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment