[SPARK-23483] Feature parity for Python vs Scala APIs

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 2.3.0
• Component/s: PySpark
• Target Version/s: 2.4.0
• Assignee: Huaxin Gao
• Reporter: Xiao Li
• Created: 21/Feb/18 19:36
• Updated: 12/Dec/22 18:11
• Resolved: 12/Sep/18 03:28

Description

Investigate the feature parity for Python vs Scala APIs and address them

Sub-Tasks

• Add other missing Catalog methods to Python API: Resolved
• spark.conf.get(value, default=None) should produce None in PySpark: Resolved
• Expose range partitioning shuffle introduced by SPARK-22614: Resolved
• Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark: Resolved
• Add KMeans distanceMeasure param to PySpark: Resolved
• Add maxDF Parameter to Python CountVectorizer: Resolved
• Add asc_nulls_first, asc_nulls_last to PySpark: Resolved
• PySpark's collect doesn't trigger QueryExecutionListener: Resolved
• Add distanceMeasure to BisectingKMeans in PySpark: Resolved