Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27544

Fix Python test script to work on Scala-2.12 build

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.0, 2.4.1, 2.4.2
    • 2.4.3
    • PySpark, Tests
    • None

    Description

      Since SPARK-27274 deprecated Scala-2.11 at Spark 2.4.1, we need to test Scala-2.12 more. This issue aims to fix the Python test script on Scala-2.12 build.

      $ dev/change-scala-version.sh 2.12
      
      $ build/sbt -Pscala-2.12 package
      
      $ python/run-tests.py --python-executables python2.7 --modules pyspark-sql
      Traceback (most recent call last):
      File "python/run-tests.py", line 69, in <module>
      raise Exception("Cannot find assembly build directory, please build Spark first.")
      Exception: Cannot find assembly build directory, please build Spark first.

      Attachments

        Issue Links

          Activity

            People

              dongjoon Dongjoon Hyun
              dongjoon Dongjoon Hyun
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: