[SPARK-35605] Move to_pandas_on_spark to the Spark DataFrame.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: PySpark
• Assignee: Haejoon Lee
• Reporter: Haejoon Lee
• Created: 02/Jun/21 01:44
• Updated: 12/Dec/22 18:10
• Resolved: 28/Jun/21 02:47

Description

Inspired by https://github.com/apache/spark/pull/32729#discussion_r643591322,

As Koalas is ported into PySpark, we don't need auto patch anymore (https://github.com/apache/spark/blob/master/python/pyspark/pandas/__init__.py#L136-L150)

Thus, we should to_pandas_on_spark and to_koalas(deprecated) to the PySpark DataFrame and add related tests.

Issue Links

• links to: [Github] Pull Request #33054 (itholic)
• links to: [Github] Pull Request #33054 (itholic)