[STORM-2844] KafkaSpout Throws IllegalStateException After Committing to Kafka When First Poll Strategy Set to EARLIEST

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0, 1.2.0
• Fix Version/s: 2.0.0, 1.2.0
• Component/s: storm-kafka-client
• Labels: pull-request-available
• Epic Link: Release Apache Storm 1.2.0
• Assignee: Hugo Da Cruz Louro
• Reporter: Hugo Da Cruz Louro
• Created: 05/Dec/17 22:23
• Updated: 27/Dec/17 23:43
• Resolved: 26/Dec/17 10:51

Description

This code, which was committed to fix STORM-2666 throws IllegalStateException when the KafkaSpout commits to Kafka and is restarted with the same consumer group id and first poll strategy is set to EARLIEST.

For example consider the following sequence:

1. KafkaSpout with consumer_group_id=TEST polls and commits offsets 1-5
2. KafkaSpout with consumer_group_id=TEST is restarted with first poll strategy set to EARLIEST

==> IllegalStateException will be thrown

This bug could be a blocker. I am setting it to Critical because assigning a different consumer id serves as a workaround to the problem.

Issue Links

• is related to: STORM-2666 Storm-kafka-client spout can sometimes emit messages that were already committed.
• links to: GitHub Pull Request #2465
• links to: GitHub Pull Request #2466