[KAFKA-8313] KafkaStreams state not being updated properly after shutdown

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: streams
• Environment: Single broker running on Ubuntu Linux.
• Assignee: Guozhang Wang
• Reporter: Eric
• Created: 02/May/19 00:01
• Updated: 10/May/19 11:47
• Resolved: 10/May/19 00:18

Description

I am running a KafkaStreams inside a DropWizard server and I am trying to detect when my stream shuts down (in case a non-recoverable error occurs).  I was hoping I could use KafkaStreams.setStateListener() to be notified when a state change occurs.  When I query the state, KafkaStreams is stuck in the REBALANCING state even though its threads are all DEAD.

 

You can easily reproduce this by doing the following:

1. Create a topic (I have one with 5 partitions)
2. Create a simple Kafka stream consuming from that topic
3. Create a StateListener and register it on that KafkaStreams
4. Start the Kafka stream
5. Once everything runs, delete the topic using kafka-topics.sh

When deleting the topic, you will see the StreamThreads' state transition from RUNNING to PARTITION_REVOKED and you will be notified with the KafkaStreams REBALANCING state.  That's all good and expected.  Then the StreamThreads transition to PENDING_SHUTDOWN and eventually to DEAD and the KafkaStreams state is stuck into the REBALANCING thread.  I was expecting to see a NOT_RUNNING state eventually... am I right?