[FLINK-1968] Make Distributed Cache more robust

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9
• Fix Version/s: 0.9
• Component/s: Runtime / Task
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 04/May/15 13:12
• Updated: 11/May/15 20:24
• Resolved: 11/May/15 20:24

Description

The distributed cache has a variety of issues at the moment.

• It does not give a proper exception when a non-cached file is accessed
• It swallows I/O exceptions that happen during file transfer and later only returns null
• It keeps inonsistently reference counts and attempts to copy often, resolving this via file collisions
• Files are not properly removes on shutdown
• No shutdown hook to remove files when process is killed