[KAFKA-5016] Consumer hang in poll method while rebalancing is in progress

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 0.10.1.0, 0.10.2.0
• Component/s: core
• Assignee: Vahid Hashemian
• Reporter: Domenico Di Giulio
• Created: 05/Apr/17 14:17
• Updated: 22/Apr/18 13:38
• Resolved: 27/Apr/17 15:11

Description

After moving to Kafka 0.10.2.0, it looks like I'm experiencing a hang in the rebalancing code.

This is a test case, not (still) production code. It does the following with a single-partition topic and two consumers in the same group:

1) a topic with one partition is forced to be created (auto-created)
2) a producer is used to write 10 messages
3) the first consumer reads all the messages and commits
4) the second consumer attempts a poll() and hangs indefinitely

The same issue can't be found with 0.10.0.0.

See the attached logs at TRACE level. Look for "SERVER HANGS" to see where the hang is found: when this happens, the client keeps failing any hearbeat attempt, as the rebalancing is in progress, and the poll method hangs indefinitely.

Issue Links

• is related to: KAFKA-3888 Allow consumer to send heartbeats in background thread (KIP-62)