[KAFKA-3523] Capture org.apache.kafka.clients.consumer.CommitFailedException in UncaughtExceptionHandler

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.10.0.0
• Fix Version/s: 0.10.0.0
• Component/s: streams
• Labels: newbie user-experience
• Assignee: Eno Thereska
• Reporter: Guozhang Wang
• Created: 07/Apr/16 01:51
• Updated: 22/Feb/17 01:04
• Resolved: 17/Apr/16 03:28

Description

When the sync commit failed due to an ongoing rebalance, it is thrown all the way up to the main thread and cause the whole Kafka Streams application to stop, even if users set UncaughtExceptionHandler. We need to be able to catch this exception in that handler as well.

Example stack trace (with UncaughtExceptionHandler set, but not been able to capture this exception):

[2016-04-06 17:49:33,891] WARN Failed to commit StreamTask #0_0 in thread [StreamThread-1]:  (org.apache.kafka.streams.processor.internals.StreamThread:485)
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:567)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:508)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:659)