Details
-
Improvement
-
Status: Resolved
-
Critical
-
Resolution: Done
-
3.1.0
-
None
-
None
Description
https://github.com/zero323/pyspark-stubs demonstrates a lot of benefits to improve usability in PySpark by leveraging Python type hints.
By having the type hints in PySpark we can, for example:
- automatically document the input and output types
- leverage IDE for error detection and auto-completion
- have a cleaner definition and easier to understand.
This JIRA is an umbrella JIRA that targets to port https://github.com/zero323/pyspark-stubs and related items to smoothly run within PySpark.
It was also discussed in the dev mailing list: http://apache-spark-developers-list.1001551.n3.nabble.com/Re-PySpark-Revisiting-PySpark-type-annotations-td26232.html
Attachments
Attachments
There are no Sub-Tasks for this issue.