[FLINK-8279] Use Mesos/YARN temp directories as fallback for BlobServer/Cache temp directories

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.5.0
• Component/s: Runtime / Coordination, Runtime / Network
• Assignee: Nico Kruber
• Reporter: Nico Kruber
• Created: 18/Dec/17 10:55
• Updated: 18/Jan/18 08:48
• Resolved: 18/Jan/18 08:46

Description

Currently, the BLOB server and cache processes (temporarily) stash incoming files into their local file system in the directory given by the blob.storage.directory configuration property. If this property is not set or empty, it will fall back to java.io.tmpdir.

Instead, in a Mesos/YARN environment, we could use the temporary directories they assigned to the Flink job which are not only the proper folder to use, but may also offer some more space.

Issue Links

• is a parent of: FLINK-8350 replace "taskmanager.tmp.dirs" with "env.io.tmp.dirs" for all components
• links to: GitHub Pull Request #5176