[SPARK-5995] Make ML Prediction Developer APIs public

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: ML
• Target Version/s: 1.4.0
• Assignee: Joseph K. Bradley
• Reporter: Joseph K. Bradley
• Created: 25/Feb/15 00:52
• Updated: 06/May/15 23:16
• Resolved: 06/May/15 23:16

Description

Previously, some Developer APIs were added to spark.ml for classification and regression to make it easier to add new algorithms and models: SPARK-4789 There are ongoing discussions about the best design of the API. This JIRA is to continue that discussion and try to finalize those Developer APIs so that they can be made public.

Please see this design doc from SPARK-4789 for details on the original API design.

Some issues under debate:

• Should there be strongly typed APIs for fit()?
  • Proposal: No
• Should the strongly typed API for transform() be public (vs. protected)?
  • Proposal: Protected for now
• What transformation methods should the API make developers implement for classification?
  • Proposal: See design doc
• Should there be a way to transform a single Row (instead of only DataFrames)?
  • Proposal: Not for now

Issue Links

• is related to: SPARK-4789 Standardize ML Prediction APIs
• is required by: SPARK-3702 Standardize MLlib classes for learners, models
• requires: SPARK-4766 ML Estimator Params should be distinct from Transformer Params
• links to: [Github] Pull Request #5913 (jkbradley)