Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6148

ClassCastException in BigQuery connector

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0.2, 1.0.0
    • Fix Version/s: 0.11.0.3, 1.0.1
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      I am trying to run a com.wepay.kafka.connect.bigquery.BigQuerySinkConnector connector, but getting the following exception.

      [2017-10-30 21:48:49,007] ERROR WorkerSinkTask

      {id=bigquery-connector-log-0} Offset commit failed, rewinding to last committed offsets (org.apache.kafka.connect.runtime.WorkerSinkTask:311)
      java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
      at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
      at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
      at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
      at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
      at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)
      [2017-10-30 21:48:49,012] ERROR Commit of WorkerSinkTask{id=bigquery-connector-log-0}

      offsets threw an unexpected exception: (org.apache.kafka.connect.runtime.WorkerSinkTask:205)
      java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
      at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
      at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
      at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
      at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
      at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
      at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:748)

      I have checked the version number of kafka client in the plug in and kafka connect itself and they are the same.

      • kafka-clients-0.11.0.0.jar matches

      I am still suspecting a type of versioning issue. Do you have any advice?

      Thanks.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kkonstantine Konstantine Karantasis
                Reporter:
                burd0047 Eugene Burd
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: