[SPARK-17822] JVMObjectTracker.objMap may leak JVM objects

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.3, 2.1.0
• Component/s: SparkR
• Target Version/s: 2.0.3, 2.1.0
• Assignee: Xiangrui Meng
• Reporter: Yin Huai
• Created: 07/Oct/16 03:52
• Updated: 23/Feb/17 19:36
• Resolved: 09/Dec/16 15:52

Description

JVMObjectTracker.objMap is used to track JVM objects for SparkR. However, we observed that JVM objects that are not used anymore are still trapped in this map, which prevents those object get GCed.

Seems it makes sense to use weak reference (like persistentRdds in SparkContext).

Issue Links

• contains: SPARK-17823 Make JVMObjectTracker.objMap thread-safe
• is related to: SPARK-17823 Make JVMObjectTracker.objMap thread-safe
• links to: [Github] Pull Request #15433 (techaddict)
• links to: [Github] Pull Request #16154 (mengxr)