[STORM-2014] New Kafka spout duplicates checking if failed messages have reached max retries

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.1.0
• Component/s: storm-kafka-client
• Epic Link: Release Apache Storm 1.1.0
• Assignee: Stig Rohde Døssing Assign to me
• Reporter: Stig Rohde Døssing
• Created: 02/Aug/16 17:15
• Updated: 12/Jul/17 06:07
• Resolved: 03/Feb/17 02:54

Description

The new Kafka spout has a RetryService interface that should make logic around retrying tuples pluggable. The RetryServiceExponentialBackoff class has code for setting a max retry count, and dropping messages once they reach the retry limit. This functionality is duplicated by the spout in the fail method, which means that the user must set different maxRetries for the RetryService and the spout in order for the RetryService code to be hit when dropping messages.

I think the retry logic belongs in the RetryService interface, and should be removed from the spout. It would also be good if the RetryService could indicate if a message will be retried or not.

Issue Links

• links to: GitHub Pull Request #1605