[FLINK-3156] FlinkKafkaConsumer fails with NPE on notifyCheckpointComplete

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.0
• Component/s: Connectors / Kafka
• Assignee: Robert Metzger
• Reporter: Till Rohrmann
• Created: 09/Dec/15 17:14
• Updated: 10/Dec/15 17:15
• Resolved: 10/Dec/15 17:15

Description

The FlinkKafkaConsumer fails with a NullPointerException when a checkpoint was completed. The stack trace is

 java.lang.RuntimeException: Error while confirming checkpoint
    at org.apache.flink.runtime.taskmanager.Task$2.run(Task.java:908)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.commitOffsets(FlinkKafkaConsumer.java:664)
    at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.notifyCheckpointComplete(FlinkKafkaConsumer.java:578)
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.notifyOfCompletedCheckpoint(AbstractUdfStreamOperator.java:176)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:546)
    at org.apache.flink.runtime.taskmanager.Task$2.run(Task.java:904)
    ... 5 more

Apparently, one of the KafkaTopicPartitionLeaders from the subscribedPartitions was null.