[SPARK-5886] Add StringIndexer

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: ML
• Target Version/s: 1.4.0
• Assignee: Xiangrui Meng
• Reporter: Xiangrui Meng
• Created: 18/Feb/15 18:56
• Updated: 12/May/15 21:42
• Resolved: 13/Apr/15 05:41

Description

`StringIndexer` takes a column of string labels (raw categories) and outputs an integer column with labels indexed by their frequency.

va li = new StringIndexer()
  .setInputCol("country")
  .setOutputCol("countryIndex")

In the output column, we should store the label to index map as an ML attribute. The index should be ordered by frequency, where the most frequent label gets index 0, to enhance sparsity.

We can discuss whether this should index multiple columns at the same time.

Issue Links

• blocks: SPARK-6113 Stabilize DecisionTree and ensembles APIs
• is blocked by: SPARK-4588 Add API for feature attributes
• is related to: SPARK-4081 Categorical feature indexing
• relates to: SPARK-7582 User guide update for StringIndexer
• links to: [Github] Pull Request #4735 (mengxr)