[FLINK-31780] Allow users to disable "Ensemble tracking" for ZooKeeper

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.18.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Oleksandr Nitavskyi
• Reporter: Oleksandr Nitavskyi
• Created: 12/Apr/23 13:55
• Updated: 15/May/23 08:06
• Resolved: 15/May/23 08:06

Description

In Apache Curator an option to skip ensemble tracking was added since version 5.0.0 (CURATOR-568)

This can be useful in certain scenarios in which CuratorFramework is accessing to ZK clusters via load balancer or Virtual IPs.
Thus in case Zookeeper of Flink user is running behind LB or Virtual IP ensemble tracking could be disabled too.

In case ZooKeeper is hidden under VIP it can return URL during Ensemble Tracking, which would lead to Unresolved Host Exception inside Curator Framework. On Flink level it would lead to cluster restart.

Currently HA with ZooKeeper can even lead to the JobManager failure. The scenario of the failure is next:

1. Flink connects to ZooKeeper via configured URL.
2. Ensemble tracking gets a new URL of ensemble, which is not obligatory accessible for Flink, because Zookeeper is under VIP.
3. In case of reconnect Flink fails to Zookeeper, moreover due to "UnresolvedHostException" Flink's jobManager is killed.

Acceptance Criteria: Users of Apache Flink has a Zookeeper config option to disable ensemble tracking for ZooKeeper.

Issue Links

• is related to: CURATOR-568 New option allowing CuratorFramework skip ZK ensemble tracking
• links to: GitHub Pull Request #22394