[FLINK-4079] YARN properties file used for per-job cluster

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.3
• Fix Version/s: 1.0.4, 1.1.0
• Component/s: Command Line Client
• Assignee: Maximilian Michels
• Reporter: Ufuk Celebi
• Created: 15/Jun/16 15:54
• Updated: 21/Jun/16 12:23
• Resolved: 21/Jun/16 12:23

Description

YARN per job clusters (flink run -m yarn-cluster) rely on the hidden YARN properties file, which defines the container configuration. This can lead to unexpected behaviour, because the per-job-cluster configuration is merged with the YARN properties file (or used as only configuration source).

A user ran into this as follows:

• Create a long-lived YARN session with HA (creates a hidden YARN properties file)
• Submits standalone batch jobs with a per job cluster (flink run -m yarn-cluster). The batch jobs get submitted to the long lived HA cluster, because of the properties file.

mxm Am I correct in assuming that this is only relevant for the 1.0 branch and will be fixed with the client refactoring you are working on?

Issue Links

• is duplicated by: FLINK-3105 Submission in per job YARN cluster mode reuses properties file of long-lived session
• is related to: FLINK-3937 Make flink cli list, savepoint, cancel and stop work on Flink-on-YARN clusters
• links to: GitHub Pull Request #2140