[SPARK-8774] Add R model formula with basic support as a transformer

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: ML, SparkR
• Target Version/s: 1.5.0
• Assignee: Eric Liang
• Reporter: Xiangrui Meng
• Shepherd: Xiangrui Meng
• Created: 02/Jul/15 00:18
• Updated: 12/Aug/15 20:51
• Resolved: 16/Jul/15 03:33

Description

To have better integration with SparkR, we can add a feature transformer to support R formula. A list of operators R supports can be find here: http://ww2.coastal.edu/kingw/statistics/R-tutorials/formulae.html

The initial version should support "~", "+", and "." on numeric columns and we can expand it in the future.

val formula = new RModelFormula()
  .setFormula("y ~ x + z")

The output should append two new columns: features and label.

Design doc is posted at https://docs.google.com/document/d/10NZNSEurN2EdWM31uFYsgayIPfCFHiuIu3pCWrUmP_c/edit?usp=sharing, as part of SPARK-6805.

Issue Links

• is related to: SPARK-9544 RFormula in Python
• relates to: SPARK-9895 User Guide for RFormula Feature Transformer
• relates to: SPARK-6805 MLlib + SparkR integration for 1.5
• links to: [Github] Pull Request #7381 (ericl)