[STORM-2544] Bugs in the Kafka Spout retry logic when using manual commit

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 2.0.0, 1.1.1, 1.2.0
• Component/s: storm-kafka-client
• Assignee: Prasanna Ranganathan
• Reporter: Prasanna Ranganathan
• Created: 07/Jun/17 13:00
• Updated: 18/Jul/17 14:26
• Resolved: 18/Jul/17 09:13

Description

Situation: Spout configured to use manual commit with a finite number of retries.

In the above scenario if and when a tuple fails repeatedly and hits the retry limit, it will neither be scheduled for an attempt again nor properly accounted for in the ack() method and in OffsetManager. This will block commits for the partition that the tuple belongs to.

Issue Links

• links to: GitHub Pull Request #2153
• links to: GitHub Pull Request #2221