[STORM-2608] Out Of Range Offsets Should Be Removed From Pending Queue

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0, 1.0.4, 1.1.1
• Fix Version/s: 2.0.0, 1.1.1, 1.2.0
• Component/s: storm-kafka
• Assignee: Kevin Conaway
• Reporter: Kevin Conaway
• Created: 29/Jun/17 20:52
• Updated: 10/Jul/17 23:10
• Resolved: 10/Jul/17 23:10

Description

There is a bug that occurs when failed tuples are invalidated due to Kafka throwing a TopicOffsetOutOfRangeException.

Below is what happens:

• Spout emits tuples
• Offsets are added to the pending tree
• Some tuples fail and are added to the failedMsgRetryManager
• On the next fetch request, a TopicOffsetOutOfRangeException is thrown and the new offset is after the offset that are currently sitting in both the pending tree and the failedMsgRetryManager
• All offsets smaller than the the new offset are removed from the failedMsgRetryManager but not the pending tree.
• Since those offsets were removed from the failedMsgRetryManager they will never be retried and thus never get removed from pending
• lastCommittedOffset() will always return the same value which means that offset in zookeeper for that partition will never get updated.

Issue Links

• links to: GitHub Pull Request #2182
• links to: GitHub Pull Request #2190
• links to: GitHub Pull Request #2195