[FLINK-14139] Fix potential memory leak of rest server when using session/standalone cluster

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.1, 1.10.0
• Component/s: Runtime / REST
• Labels: pull-request-available
• Assignee: Yingjie Cao
• Reporter: Yingjie Cao
• Created: 20/Sep/19 07:03
• Updated: 25/Sep/19 14:21
• Resolved: 25/Sep/19 14:21

Description

Flink's rest server uses netty decoder for http request processing and file uploading. However io.netty.handler.codec.http.multipart.DiskAttribute and io.netty.handler.codec.http.multipart.DiskFileUpload class of netty would register some temp files, including post chunks and upload file chunks, to java.io.DeleteOnExitHook which has a potential of memory leak, because the registered file names will never be deleted before the cluster stops. 

Most of the time, this is not a big problem, however we use Flink session cluster a long running service for ad-hoc SQL query, this problem gets worse.

In fact, Flink handles the clean up of temp files through org.apache.flink.util.ShutdownHookUtil (though not including the post chunks), so it is no need to register these files to java.io.DeleteOnExitHook.

Issue Links

• links to: GitHub Pull Request #9750