Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27544

Fix Python test script to work on Scala-2.12 build

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 2.4.1, 2.4.2
    • Fix Version/s: 2.4.3
    • Component/s: PySpark, Tests
    • Labels:
      None

      Description

      Since SPARK-27274 deprecated Scala-2.11 at Spark 2.4.1, we need to test Scala-2.12 more. This issue aims to fix the Python test script on Scala-2.12 build.

      $ dev/change-scala-version.sh 2.12
      
      $ build/sbt -Pscala-2.12 package
      
      $ python/run-tests.py --python-executables python2.7 --modules pyspark-sql
      Traceback (most recent call last):
      File "python/run-tests.py", line 69, in <module>
      raise Exception("Cannot find assembly build directory, please build Spark first.")
      Exception: Cannot find assembly build directory, please build Spark first.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dongjoon Dongjoon Hyun
                Reporter:
                dongjoon Dongjoon Hyun
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: