[SPARK-33247] Improve examples and scenarios in docstrings - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.1.0
Fix Version/s: None
Component/s: Documentation, PySpark
Labels:
None

Epic Link:
Project Zen

Description

Currently, PySpark documentation does not have a lot of examples and scenarios. See also https://github.com/apache/spark/pull/30149#issuecomment-716490037.

We should add/improve examples especially in the commonly used APIs. For example, Column, DataFrame. RDD, SparkContext, etc.

This umbrella JIRA targets to improve them in commonly used APIs.

NOTE that we'll have to convert the docstrings into numpydoc style first in a separate PR (at ~~SPARK-32085~~), and then add examples. In this way, we can manage migration to numpydoc and example improvement here separately (e.g., reverting numpydoc migration only).