[KAFKA-9128] ConsumerCoordinator ignores exceptions from ConsumerRebalanceListener

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.3.0
• Component/s: consumer
• Assignee: Unassigned
• Reporter: Oleg Muravskiy
• Created: 01/Nov/19 08:31
• Updated: 01/Nov/19 16:53

Description

I'm using a custom ConsumerRebalanceListener with a plain old Kafka consumer to manage offsets in an external storage as described in ConsumerRebalanceListener's javadoc.

When that storage is not available, I'm throwing an exception from my listener. However, the exception is simply logged and ignored by the ConsumerCoordinator, as could be seen in these two code snippets from it:

       try {
           listener.onPartitionsRevoked(revoked);
       } catch (WakeupException | InterruptException e) {
           throw e;
       } catch (Exception e) {
           log.error("User provided listener {} failed on partition revocation", listener.getClass().getName(), e);
       }

and

       try {
           listener.onPartitionsAssigned(assignedPartitions);
       } catch (WakeupException | InterruptException e) {
           throw e;
       } catch (Exception e) {
           log.error("User provided listener {} failed on partition assignment", listener.getClass().getName(), e);
       }

The consumption continues as if nothing has happened.