[KAFKA-13270] Kafka may fail to connect to ZooKeeper, retry forever, and never start

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Assignee: Ismael Juma
• Reporter: Ron Dagostino
• Created: 03/Sep/21 01:12
• Updated: 06/Sep/21 16:21
• Resolved: 06/Sep/21 16:21

Description

The implementation of https://issues.apache.org/jira/browse/ZOOKEEPER-3593 in ZooKeeper version 3.6.0 decreased the default value for the ZooKeeper client's `jute.maxbuffer` configuration from 4MB to 1MB. This can cause a problem if Kafka tries to retrieve a large amount of data across many znodes – in such a case the ZooKeeper client will repeatedly emit a message of the form "java.io.IOException: Packet len <####> is out of range" and the Kafka broker will never connect to ZooKeeper and fail to make progress on the startup sequence. We can avoid the potential for this issue to occur by explicitly setting the value to 4MB whenever we create a new ZooKeeper client as long as no explicit value has been set via the `jute.maxbuffer` system property.

Issue Links

• links to: GitHub Pull Request #11295