Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5535

Transformations - tranformations for value broken on tombstone events

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.2.1
    • Fix Version/s: 1.0.0
    • Component/s: KafkaConnect
    • Labels:
    • Environment:
      Unbuntu 14.04
      Java 8

      Description

      I'm trying to use the transformation for Kafka Connect and running into issues.

      The tranformation configuration is:
      -----------------
      "transforms": "GetAfter",
      "transforms.GetAfter.type": "org.apache.kafka.connect.transforms.ExtractField$Value",
      "transforms.GetAfter.field": "after",
      -----------------
      And I got the following errors occasionally´╝Ü
      -----------------
      org.apache.kafka.connect.errors.DataException: Only Map objects supported in absence of schema for [field extraction], found: null
      at org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
      at org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:57)
      at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:39)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:408)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:249)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:179)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
      at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:139)
      at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:182)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      -----------------
      Seems that tombstone events break the transformation, but after checking the source code for Transformations on Value (ExtractField$Value, ValueToKey, MaskField$Value, ReplaceField$Value), none of them handles tombstone events explicitly, none of them work through Tombstone events neither.
      Null check in those transformations may be nessesary.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ewencp Ewen Cheslack-Postava
                Reporter:
                alei Yelei Wu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: