[FLINK-6523] Improve lifecycle management of created BlobCaches in TaskManagerLogHandler and JobClient

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Affects Version/s: 1.3.0, 1.4.0
• Component/s: Runtime / Coordination
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 10/May/17 11:08
• Updated: 11/Oct/19 09:33
• Resolved: 11/Oct/19 09:33

Description

Some components like the JobClient#retrieveClassLoader and TaskManagerLogHandler create BlobCaches to retrieve data from the BlobServer. These instances are created in a local context and their lifecycle is not properly managed. The cleanup/closing solely relies on the fact that their storage directory will be cleaned up eventually by a registered shutdown hook.

I think it would be better to properly implement a lifecycle management for these components and close them at an appropriate location in the code. E.g. one could pass in a shared BlobCache which is closed once the JobClient action is over.