Description
Currently, the minimum supported Pandas version is 0.19.2. We bumped up testing in the Jenkins env to 0.23.2 and since 0.19.2 was released nearly 3 years ago, it is not always compatible with other Python libraries. Increasing the version to 0.23.2 will also allow some workarounds to be removed and make maintenance easier.
Attachments
Issue Links
- is related to
-
SPARK-28003 spark.createDataFrame with Arrow doesn't work with pandas.NaT
-
- Resolved
-
- links to