[SYSTEMDS-2234] Memory leak buffer pool due to missing variable cleanup

• Type: Task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: SystemML 1.2
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 05/Apr/18 21:08
• Updated: 06/Apr/18 21:09
• Resolved: 06/Apr/18 21:09

Description

There are scenarios where variables are not properly cleaned up before assigning new matrices to logical variable names. This missing cleanup leads to a memory leak in the buffer pool, which in turn leads to unnecessary evictions and missing cleanup of FS and HDFS files.

After a closer investigation, this issue occurs for special cases of function calls that have a peculiar control flow structure that prevents rmvar instructions for certain intermediates and thus requires cleanup of these intermediates after the function call. Furthermore, a similar issue occurs for loops with dedicated update-in place variables.