[SPARK-12808] Formula based GLM in PySpark

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Component/s: MLlib, PySpark
• Labels: GLM mllib pyspark
• Assignee: Unassigned
• Reporter: Shubhanshu Mishra
• Shepherd: Shivram Mani
• Created: 13/Jan/16 17:08
• Updated: 22/Feb/16 08:32
• Resolved: 22/Feb/16 08:32

Description

I believe PySpark's mllib module should support a GLM feature with also includes defining models using a formula. This is done in a python package called statsmodels http://statsmodels.sourceforge.net/devel/example_formulas.html

The formula feature can be implemented using the python module patsy.

Currently, RSpark supports a GLM module with formula feature.

I can give a shot implementing the feature.

Issue Links

• Is contained by: SPARK-11106 Should ML Models contains single models or Pipelines?