[FLINK-9762] CoreOptions.TMP_DIRS wrongly managed on Yarn

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.2, 1.6.0
• Component/s: Deployment / YARN
• Labels: pull-request-available
• Assignee: Oleksandr Nitavskyi
• Reporter: Oleksandr Nitavskyi
• Created: 05/Jul/18 10:51
• Updated: 24/Jul/18 16:34
• Resolved: 19/Jul/18 11:42

Description

The issue on Yarn is that it is impossible to have different LOCAL_DIRS on JobManager and TaskManager, despite LOCAL_DIRS value depends on the container.

The issue is that CoreOptions.TMP_DIRS is configured to the default value during JobManager initialization and added to the configuration object. When TaskManager is launched the appropriate configuration object is cloned with LOCAL_DIRS which makes sense only for Job Manager container. When YARN container with TaskManager from his point of view CoreOptions.TMP_DIRS is always equal either to path in flink.yml or to the or to the LOCAL_DIRS of Job Manager (default behaviour). Is TaskManager’s container do not have an access to another folders, that folders allocated by YARN TaskManager cannot be started.

Issue Links

• causes: FLINK-9939 Mesos: Not setting TMP dirs causes NPE
• links to: GitHub Pull Request #6284