[SPARK-18822] Support ML Pipeline in SparkR

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: ML, SparkR
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Felix Cheung
• Shepherd: Felix Cheung
• Created: 11/Dec/16 19:07
• Updated: 08/Oct/19 05:41
• Resolved: 08/Oct/19 05:41

Description

From Joseph Bradley:

"
Supporting Pipelines and advanced use cases: There really needs to be more design discussion around SparkR. Felix Cheung would you be interested in leading some discussion? I'm envisioning something similar to what was done a while back for Pipelines in Scala/Java/Python, where we consider several use cases of MLlib: fitting a single model, creating and tuning a complex Pipeline, and working with multiple languages. That should help inform what APIs should look like in Spark R.
"

Certain ML model, such as OneVsRest, is harder to represent in a single call R API. Having advanced API or Pipeline API like this could help to expose that to our users.

Issue Links

• is related to: SPARK-24359 SPIP: ML Pipelines in R