[SYSTEMDS-1430] Robust memory handling broadcasts

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: SystemML 0.14
• Component/s: APIs, Runtime
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 22/Mar/17 02:46
• Updated: 21/Apr/17 05:05
• Resolved: 23/Mar/17 20:42

Description

This task aims to make the memory handling of re-used and temporary broadcast variables more robust in order to avoid unnecessary OOMs.

1) Explicitly destroy temporary broadcast variables because we would otherwise never clean them up and Spark's ContextCleaner seems to be a best effort daemon that triggers every 30min or on garbage collection (which might be too late if a large object is allocated)

2) Keep track of currently softly reachable (re-used) broadcasts in order to take their size into account when deciding on guarded collect.