[SPARK-36288] Update API usage on pyspark pandas documents

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: Documentation, PySpark
• Target Version/s: 3.2.0
• Assignee: Leona Yoda
• Reporter: Leona Yoda
• Created: 26/Jul/21 11:03
• Updated: 12/Dec/22 18:10
• Resolved: 27/Jul/21 03:31

Description

I found several warning messages when I tested around ported Spark Pandas API Documents (https://issues.apache.org/jira/browse/SPARK-34885).

1.  `spark.sql.execution.arrow.enabled` on Best Practice document

21/07/26 05:42:02 WARN SQLConf: The SQL config 'spark.sql.execution.arrow.enabled' has been deprecated in Spark v3.0 and may be removed in the future. Use 'spark.sql.execution.arrow.pyspark.enabled' instead of it.

 

2. `DataFrame.to_spark_io` on From/to other DBMSes document

/opt/spark/python/lib/pyspark.zip/pyspark/pandas/frame.py:4811: FutureWarning: Deprecated in 3.2, Use spark.to_spark_io instead. warnings.warn("Deprecated in 3.2, Use spark.to_spark_io instead.", FutureWarning)

 

At this time it worked but I think it's better to update API usage on those documents.

Issue Links

• links to: [Github] Pull Request #33519 (yoda-mon)