[SPARK-24225] Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.6.3, 2.2.0, 2.2.1, 2.3.0
• Component/s: Block Manager, Spark Core
• Labels: bulk-closed
• Assignee: Doug Rohrer
• Reporter: Doug Rohrer
• Shepherd: DB Tsai
• Created: 09/May/18 18:51
• Updated: 25/May/21 01:54
• Resolved: 25/May/21 01:38

Description

When using Broadcast Variables, it would be beneficial if classes implementing AutoClosable were closed when released. This would allow broadcast variables to be used, for example, as shared resource pools across multiple tasks within an executor without the use of static variables. 

Issue Links

• links to: [Github] Pull Request #21322 (JeetKunDoug)