[STORM-2184] New Kafka spout will fail to commit offsets on shutdown if there are offsets to commit

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.1.0, 1.0.3
• Component/s: storm-kafka-client
• Assignee: Stig Rohde Døssing
• Reporter: Stig Rohde Døssing
• Created: 02/Nov/16 17:32
• Updated: 12/Jul/17 06:08
• Resolved: 07/Nov/16 14:19

Description

As far as I know spout methods are not called concurrently by Storm. When the new Kafka spout closes or deactivates, the closing thread calls KafkaConsumer.wakeup. If there are uncommitted offsets and the spout is in manual commit mode, it then tries to commit offsets. Since there are no other threads concurrently interacting with the spout, the call to KafkaConsumer.commitSync will end up throwing WakeupException. This is because the behavior of wakeup is to cause the next blocking call to throw WakeupException if there are no other threads blocking on the consumer when wakeup is called.

The call to wakeup should be removed, since it does nothing if there are no offsets to commit, and causes an uncaught exception if there are.

Issue Links

• links to: GitHub Pull Request #1761