Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32010

Thread leaks in pinned thread mode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.1.0
    • Component/s: PySpark, SQL
    • Labels:
      None

      Description

      SPARK-22340 introduced a pin thread mode which guarantees you to sync Python thread and JVM thread.

      However, looks like the JVM threads are not finished even when the Python thread is finished. It can be debugged via YourKit, and run multiple jobs with multiple threads at the same time.

      Easiest reproducer is:

      PYSPARK_PIN_THREAD=true ./bin/pyspark
      
      >>> from threading import Thread
      >>> Thread(target=lambda: spark.range(1000).collect()).start()
      >>> Thread(target=lambda: spark.range(1000).collect()).start()
      >>> Thread(target=lambda: spark.range(1000).collect()).start()
      >>> spark._jvm._gateway_client.deque
      deque([<py4j.clientserver.ClientServerConnection object at 0x119f7aba8>, <py4j.clientserver.ClientServerConnection object at 0x119fc9b70>, <py4j.clientserver.ClientServerConnection object at 0x119fc9e10>, <py4j.clientserver.ClientServerConnection object at 0x11a015358>, <py4j.clientserver.ClientServerConnection object at 0x119fc00f0>])
      >>> Thread(target=lambda: spark.range(1000).collect()).start()
      >>> spark._jvm._gateway_client.deque
      deque([<py4j.clientserver.ClientServerConnection object at 0x119f7aba8>, <py4j.clientserver.ClientServerConnection object at 0x119fc9b70>, <py4j.clientserver.ClientServerConnection object at 0x119fc9e10>, <py4j.clientserver.ClientServerConnection object at 0x11a015358>, <py4j.clientserver.ClientServerConnection object at 0x119fc08d0>, <py4j.clientserver.ClientServerConnection object at 0x119fc00f0>])
      

      The connection doesn't get closed, and it holds JVM thread running.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hyukjin.kwon Hyukjin Kwon
                Reporter:
                hyukjin.kwon Hyukjin Kwon
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: