[SPARK-6805] MLlib + SparkR integration for 1.5

• Type: Umbrella
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: ML, SparkR
• Target Version/s: 1.5.0
• Assignee: Eric Liang
• Reporter: Xiangrui Meng
• Created: 09/Apr/15 17:03
• Updated: 07/Aug/15 16:44
• Resolved: 05/Aug/15 17:51

Description

-SparkR was merged. So let's have this umbrella JIRA for the ML pipeline API in SparkR. The implementation should be similar to the pipeline API implementation in Python.-

We limited the scope of this JIRA to MLlib + SparkR integration for 1.5.

For Spark 1.5, we want to support linear/logistic regression in SparkR, with basic support for R formula and elastic-net regularization. The design doc can be viewed at https://docs.google.com/document/d/10NZNSEurN2EdWM31uFYsgayIPfCFHiuIu3pCWrUmP_c/edit?usp=sharing

Issue Links

• incorporates: SPARK-9463 Expose model coefficients with names in SparkR RFormula
• incorporates: SPARK-9201 Integrate MLlib with SparkR using RFormula
• incorporates: SPARK-9230 SparkR RFormula should support StringType features
• incorporates: SPARK-9391 Support minus, dot, and intercept operators in SparkR RFormula
• incorporates: SPARK-9713 Document SparkR MLlib glm() integration in Spark 1.5
• is depended upon by: SPARK-9647 MLlib + SparkR integration for 1.6
• is related to: SPARK-8774 Add R model formula with basic support as a transformer
• relates to: SPARK-4586 Python API for ML Pipeline
• links to: [Github] Pull Request #7483 (ericl)