[SPARK-23109] ML 2.3 QA: API: Python API coverage

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Done
• Affects Version/s: 2.3.0
• Component/s: Documentation, ML, PySpark
• Target Version/s: 2.3.0
• Assignee: Bryan Cutler
• Reporter: Joseph K. Bradley
• Created: 17/Jan/18 00:33
• Updated: 29/Jan/18 17:26
• Resolved: 29/Jan/18 17:23

Description

For new public APIs added to MLlib (spark.ml only), we need to check the generated HTML doc and compare the Scala & Python versions.

• GOAL: Audit and create JIRAs to fix in the next release.
• NON-GOAL: This JIRA is not for fixing the API parity issues.

We need to track:

• Inconsistency: Do class/method/parameter names match?
• Docs: Is the Python doc missing or just a stub? We want the Python doc to be as complete as the Scala doc.
• API breaking changes: These should be very rare but are occasionally either necessary (intentional) or accidental. These must be recorded and added in the Migration Guide for this release.
  • Note: If the API change is for an Alpha/Experimental/DeveloperApi component, please note that as well.
• Missing classes/methods/parameters: We should create to-do JIRAs for functionality missing from Python, to be added in the next release cycle. Please use a separate JIRA (linked below as "requires") for this list of to-do items.

Issue Links

• requires: SPARK-22005 CrossValidator, TrainValidationSplit dump sub models to disk when fitting: Python API
• requires: SPARK-22796 Add multiple column support to PySpark QuantileDiscretizer
• requires: SPARK-22797 Add multiple column support to PySpark Bucketizer
• requires: SPARK-21741 Python API for DataFrame-based multivariate summarizer
• requires: SPARK-23161 Add missing APIs to Python GBTClassifier
• requires: SPARK-23162 PySpark ML LinearRegressionSummary missing r2adj
• requires: SPARK-23256 Add columnSchema method to PySpark image reader
• requires: SPARK-23163 Sync Python ML API docs with Scala