[FLINK-21382] Standalone K8s documentation does not explain usage of standby JobManagers

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.1, 1.13.0
• Fix Version/s: 1.13.0, 1.12.3
• Component/s: Deployment / Kubernetes, Documentation
• Labels: pull-request-available
• Assignee: Yang Wang
• Reporter: Till Rohrmann
• Created: 16/Feb/21 16:17
• Updated: 23/Mar/21 18:05
• Resolved: 23/Mar/21 18:05

Description

Our standalone K8s documentation mentions how to configure K8s HA services. It does not mention that this only works with a single JobManager. When using standby JobManagers, then the given deployment yamls won't work because the jobmanager.rpc.address is configured to be the jobmanager service.

Changing the configuration to work is surprisingly difficult because of a lack of documentation. Moreover, it is quite difficult to pass in custom configuration values when using a ConfigMap for sharing Flink's flink-conf.yaml. The problem is that mounted ConfigMaps are not writable from a pod perspective. See this answer for how one could achieve it.

I think we could improve our documentation to explain our users how to configure a standalone HA cluster with standby JobManagers.

Issue Links

• is duplicated by: FLINK-21082 Document how to start multiple JobManagers in HA mode for Kubernetes deployment
• is related to: FLINK-21432 Web UI -- Error - {"errors":["Service temporarily unavailable due to an ongoing leader election. Please refresh."]}
• is related to: FLINK-21685 Flink JobManager failed to restart from checkpoint in kubernetes HA setup
• is related to: FLINK-21472 FencingTokenException: Fencing token mismatch
• links to: GitHub Pull Request #15248