Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-41285 Test basework and improvement of test coverage in Spark Connect
  3. SPARK-42272

Use available ephemeral port for Spark Connect server in testing

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Connect, Tests
    • None

    Description

      Currently Spark Connect tests cannot run in parallel, and requires to set the parallelism as 1

      python/run-tests --module pyspark-connect --parallelism 1
      

      The main reason is because of the port being used is hardcorded as the default 15002. We should better search available port, and use it

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            gurwls223 Hyukjin Kwon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: