[STORM-2648] Kafka spout can't show acks/fails and complete latency when auto commit is enabled

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0, 1.1.0, 1.2.0
• Fix Version/s: 2.0.0, 1.2.0
• Component/s: storm-kafka-client
• Labels: pull-request-available
• Epic Link: Release Apache Storm 1.2.0
• Assignee: Stig Rohde Døssing
• Reporter: Stig Rohde Døssing
• Created: 20/Jul/17 10:14
• Updated: 02/Oct/17 16:10
• Resolved: 02/Oct/17 16:10

Description

The storm-kafka-client spout currently emits tuples with no message ids if auto commit is enabled. This causes the ack/fail/complete latency counters in Storm UI to be 0. In some cases this is desirable because the user may not care, and doesn't want the overhead of Storm tracking tuples. avermeerbergen expressed a desire to be able to use auto commit without these counters being disabled, presumably to monitor topology performance.

We should add a toggle that allows users to enable/disable tuple anchoring in the auto commit case.

Issue Links

• relates to: STORM-2357 add At-Most-Once guarantee in KafkaSpout
• links to: GitHub Pull Request #2249
• links to: GitHub Pull Request #2353