[SPARK-31852] Redesign PySpark documentation

• Type: Umbrella
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 3.1.0
• Component/s: ML, PySpark, Spark Core, SQL, Structured Streaming
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 28/May/20 06:13
• Updated: 12/Dec/22 17:51
• Resolved: 28/May/20 06:14

Description

Currently, PySpark documentation (https://spark.apache.org/docs/latest/api/python/index.html) is pretty much poorly written compared to other projects.
See, for example, see pandas https://pandas.pydata.org/docs/

PySpark is being more and more important in Spark, and we should improve this documentation so people can easily follow.

Reference:

• https://pandas.pydata.org/docs/
• https://koalas.readthedocs.io/en/latest/

Issue Links

• duplicates: SPARK-31851 Redesign PySpark documentation