[STORM-1674] Idle KafkaSpout consumes more bandwidth than needed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.3, 0.10.1, 1.0.1
• Fix Version/s: 2.0.0, 1.0.2, 1.1.0
• Component/s: storm-kafka
• Labels: pull-request-available
• Epic Link: Release Apache Storm 1.0.2
• Assignee: Robert Hastings
• Reporter: Robert Hastings
• Created: 01/Apr/16 03:35
• Updated: 22/Oct/18 19:26
• Resolved: 20/May/16 20:04

Description

Discovered 30 megabits of traffic flowing between a set of KafkaSpouts
and our kafka servers even though no Kafka messages were moving.
Using the wireshark kafka dissector, we were able to see that
each FetchRequest had maxWait set to 10000
and minBytes set to 0. When binBytes is set to 0 the kafka server
responds immediately when there are no messages. In turn the KafkaSpout
polls without any delay causing a constant stream of FetchRequest/
FetchResponse messages. Using a non-KafkaSpout client had a similar
traffic pattern with two key differences
1) minBytes was 1
2) maxWait was 100
With these FetchRequest parameters and no messages flowing,
the kafka server delays the FetchResponse by 100 ms. This reduces
the network traffic from megabits to the low kilobits. It also
reduced the CPU utilization of our kafka server from 140% to 2%.

Issue Links

• links to: GitHub Pull Request #1287
• links to: GitHub Pull Request #1391
• links to: GitHub Pull Request #1393