[SEDONA-14] Saving dataframe to CSV or Parquet fails due to unknown type

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.1
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Hugh Saalmans
• Created: 29/Jan/21 04:14
• Updated: 09/Feb/21 22:27
• Resolved: 09/Feb/21 22:26

Description

Tested on Pyspark 3.0.1 (Python 3.8.6) with Sedona 1.0.0 RC1 on a Macbook Pro (Big Sur) and Pyspark 3.0.1 (Python 3.7.9) with Sedona 1.0.0 RC1 on AWS EMR 6.2.0.

After adding a point geometry field to a dataframe, I'm unable to save the new dataframe. Saving to CSV returns a Python error (attached); whilst saving to Parquet returns the Java stack trace attached:

Test file also attached.

Issue Links

• links to: GitHub Pull Request #508