Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3523

Capture org.apache.kafka.clients.consumer.CommitFailedException in UncaughtExceptionHandler

    Details

      Description

      When the sync commit failed due to an ongoing rebalance, it is thrown all the way up to the main thread and cause the whole Kafka Streams application to stop, even if users set UncaughtExceptionHandler. We need to be able to catch this exception in that handler as well.

      Example stack trace (with UncaughtExceptionHandler set, but not been able to capture this exception):

      [2016-04-06 17:49:33,891] WARN Failed to commit StreamTask #0_0 in thread [StreamThread-1]:  (org.apache.kafka.streams.processor.internals.StreamThread:485)
      org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
          at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:567)
          at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:508)
          at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:659)
      

        Attachments

          Activity

            People

            • Assignee:
              enothereska Eno Thereska
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: