[STORM-3067] Kafka Spout has no active members in consumer group

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.1.2
• Component/s: storm-kafka-client
• Assignee: Unassigned
• Reporter: Rui Abreu
• Created: 10/May/18 09:45
• Updated: 11/May/18 08:00
• Resolved: 11/May/18 07:57

Description

Using Kafka 10.2.1, two different behaviours were observed:

Storm 1.1.0 =>

/opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group test.topic.consumer-group --describe
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).


TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
test.topic 0 85186604 85186607 3 consumer-2-27a77d1b-e851-47a4-954e-4953ea612b72 /X.X.X.X consumer-2

 

Storm 1.1.2 =>

/opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group test.topic.consumer-group --describe
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).



Consumer group 'test.topic.consumer-group' has no active members.

TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
test.topic 0 85202473 85202475 2 -

Despite this behaviour, the topology continues to consume messages and commit offsets to Kafka. It's unclear if this lack of active clients on the consumer group affects the normal functioning of the Spout.