[FLINK-21383] Docker image does not play well together with ConfigMap based flink-conf.yamls

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.11.6, 1.12.7, 1.13.5, 1.14.3
• Component/s: Deployment / Kubernetes, flink-docker
• Labels: auto-deprioritized-major auto-deprioritized-minor usability
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 16/Feb/21 16:25
• Updated: 24/Aug/23 10:35

Description

Flink's Docker image does not play well together with ConfigMap based flink-conf.yamls. The docker-entrypoint.sh script offers a few env variables to overwrite configuration values (e.g. FLINK_PROPERTIES, JOB_MANAGER_RPC_ADDRESS, etc.). The problem is that the entrypoint script assumes that it can modify the existing flink-conf.yaml. This is not the case if the flink-conf.yaml is based on a ConfigMap.

Making things worse, failures updating the flink-conf.yaml are not reported. Moreover, the called jobmanager.sh and taskmanager.sh scripts don't support to pass in dynamic configuration properties into the processes.

I think the problem is that our assumption that we can modify the flink-conf.yaml does not always hold true. If we updated the final configuration from within the Flink process (dynamic properties and env variables), then this problem could be avoided.

Issue Links

• causes: FLINK-29145 docker-entrypoint.sh - Read-only file system
• relates to: FLINK-24474 Standalone clusters should bind to localhost by default