[KAFKA-6148] ClassCastException in BigQuery connector

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0.2, 1.0.0
• Fix Version/s: 0.11.0.3, 1.0.1
• Component/s: connect
• Assignee: Konstantine Karantasis
• Reporter: Eugene Burd
• Created: 30/Oct/17 22:01
• Updated: 30/Jan/18 16:49
• Resolved: 30/Jan/18 16:49

Description

I am trying to run a com.wepay.kafka.connect.bigquery.BigQuerySinkConnector connector, but getting the following exception.

[2017-10-30 21:48:49,007] ERROR WorkerSinkTask

{id=bigquery-connector-log-0} Offset commit failed, rewinding to last committed offsets (org.apache.kafka.connect.runtime.WorkerSinkTask:311)
java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
[2017-10-30 21:48:49,012] ERROR Commit of WorkerSinkTask{id=bigquery-connector-log-0}

offsets threw an unexpected exception: (org.apache.kafka.connect.runtime.WorkerSinkTask:205)
java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

I have checked the version number of kafka client in the plug in and kafka connect itself and they are the same.

• kafka-clients-0.11.0.0.jar matches

I am still suspecting a type of versioning issue. Do you have any advice?

Thanks.

Issue Links

• links to: GitHub Pull Request #4457