Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27544

Fix Python test script to work on Scala-2.12 build

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 2.4.1, 2.4.2
    • Fix Version/s: 2.4.3
    • Component/s: PySpark, Tests
    • Labels:
      None

      Description

      Since SPARK-27274 deprecated Scala-2.11 at Spark 2.4.1, we need to test Scala-2.12 more. This issue aims to fix the Python test script on Scala-2.12 build.

      $ dev/change-scala-version.sh 2.12
      
      $ build/sbt -Pscala-2.12 package
      
      $ python/run-tests.py --python-executables python2.7 --modules pyspark-sql
      Traceback (most recent call last):
      File "python/run-tests.py", line 69, in <module>
      raise Exception("Cannot find assembly build directory, please build Spark first.")
      Exception: Cannot find assembly build directory, please build Spark first.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              dongjoon Dongjoon Hyun
              Reporter:
              dongjoon Dongjoon Hyun

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment