[KAFKA-4041] kafka unable to reconnect to zookeeper behind an ELB

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.9.0.1, 0.10.0.1
• Fix Version/s: 2.0.0
• Environment: RHEL EC2 instances
• Assignee: Ismael Juma
• Reporter: prabhakar
• Created: 15/Aug/16 18:02
• Updated: 16/Oct/18 18:20
• Resolved: 17/Jul/18 20:54

Description

Kafka brokers are unable to connect to zookeeper which is behind an ELB.

Kafka is using zkClient which is caching the IP address of zookeeper and even when there is a change in the IP for zookeeper it is using the Old zookeeper IP.

The server.properties has a DNS name. Ideally kafka should resolve the IP using the DNS in case of any failures connecting to the broker.

Issue Links

• duplicates: KAFKA-5473 handle ZK session expiration properly when a new session can't be established
• is caused by: ZOOKEEPER-2184 Zookeeper Client should re-resolve hosts when connection attempts fail
• is duplicated by: KAFKA-4871 Kafka doesn't respect TTL on Zookeeper hostname - crash if zookeeper IP changes
• links to: GitHub Pull Request #5376