Uploaded image for project: 'Apache Sedona'
  1. Apache Sedona
  2. SEDONA-128

Change Sedona python code to support pyspark 3.3

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • None
    • None

    Description

      There are some changes in pyspark 3.3 that affect Sedona.

      • PickleSerializer was replaced with CPickleSerializer
      • SparkSession._wrapped: SQLContext is removed

       

      Attachments

        Issue Links

          is fixed by

          Improvement - An improvement or enhancement to an existing feature or task. SEDONA-129 Bump Spark version to 3.3.0

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. SEDONA-150 Drop Spark 2.4 and Scala 2.11 support

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #635

          Activity

            People

              Unassigned Unassigned
              umartin Martin Andersson
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m