[SPARK-6823] Add a model.matrix like capability to DataFrames (modelDataFrame)

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: ML, SparkR
• Labels: bulk-closed
• External issue URL: https://sparkr.atlassian.net/browse/SPARKR-197
• Assignee: Unassigned
• Reporter: Shivaram Venkataraman
• Created: 09/Apr/15 21:30
• Updated: 21/May/19 04:34
• Resolved: 21/May/19 04:34

Description

Currently Mllib modeling tools work only with double data. However, data tables in practice often have a set of categorical fields (factors in R), that need to be converted to a set of 0/1 indicator variables (making the data actually used in a modeling algorithm completely numeric). In R, this is handled in modeling functions using the model.matrix function. Similar functionality needs to be available within Spark.