Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17822

JVMObjectTracker.objMap may leak JVM objects

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.3, 2.1.0
    • Component/s: SparkR
    • Labels:
      None

      Description

      JVMObjectTracker.objMap is used to track JVM objects for SparkR. However, we observed that JVM objects that are not used anymore are still trapped in this map, which prevents those object get GCed.

      Seems it makes sense to use weak reference (like persistentRdds in SparkContext).

        Attachments

        1. screenshot-1.png
          863 kB
          Yin Huai

          Issue Links

            Activity

              People

              • Assignee:
                mengxr Xiangrui Meng
                Reporter:
                yhuai Yin Huai
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: