[FLINK-25764] Docker sets JobManager's rpc address to same host by default

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.15.0
• Component/s: Deployment / Kubernetes, flink-docker
• Labels: auto-deprioritized-minor usability
• Assignee: Unassigned
• Reporter: Niklas Semmler
• Created: 23/Jan/22 15:25
• Updated: 31/Jul/22 22:37

Description

In the docker entrypoint, the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25). This environment variable overrides the value set for jobmanager.rpc.address in the flink config (line 78, 71). For the TaskManager, this means that it tries to find the JobManager on the same host. When this is not the case, the TaskManager will retry and ultimately terminate. Hence, for cluster deployments, the variable has to be defined when starting the docker.

For Kubernetes deployments, the TaskManager cannot connect to the jobmanager.rpc.address even when it is defined by the flink configmap. However, we don't see this problem pop up, because for now the configmap is mounted read-only into the containers (see FLINK-21383 for more details).

To simplify this configuration, I propose to (a) never set a default setting for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any non-JobManager container. The only down-side is that all docker deployments will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and JobManager run on the same node.