[KAFKA-4306] Connect workers won't shut down if brokers are not available

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.1.0
• Fix Version/s: 0.10.2.0
• Component/s: connect
• Assignee: Konstantine Karantasis
• Reporter: Gwen Shapira
• Created: 17/Oct/16 16:50
• Updated: 06/Dec/16 22:35
• Resolved: 06/Dec/16 22:35

Description

If brokers are not available and we try to shut down connect workers, sink connectors will be stuck in a loop retrying to commit offsets:

2016-10-17 09:39:14,907] INFO Marking the coordinator 192.168.1.9:9092 (id: 2147483647 rack: null) dead for group connect-dump-kafka-config1 (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:600)
[2016-10-17 09:39:14,907] ERROR Commit of WorkerSinkTask

{id=dump-kafka-config1-0}

offsets threw an unexpected exception: (org.apache.kafka.connect.runtime.WorkerSinkTask:194)
org.apache.kafka.clients.consumer.RetriableCommitFailedException: Offset commit failed with a retriable exception. You should retry committing offsets.
Caused by: org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException

We should probably limit the number of retries before doing "unclean" shutdown.

Issue Links

• is related to: KAFKA-4154 Kafka Connect fails to shutdown if it has not completed startup
• links to: GitHub Pull Request #2201