[SPARK-11215] Add multiple columns support to StringIndexer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: ML

Docs Text

When specifying frequencyDesc or frequencyAsc as stringOrderType param in StringIndexer, in case of equal frequency, the order of strings was previously undefined. Since Spark 3.0, strings with equal frequency are further
sorted lexicographically.

• Assignee: L. C. Hsieh
• Reporter: Yanbo Liang
• Created: 20/Oct/15 15:18
• Updated: 08/Jun/20 17:38
• Resolved: 29/Jan/19 15:21

Description

Add multiple columns support to StringIndexer, then users can transform multiple input columns to multiple output columns simultaneously. See discussion SPARK-8418.

Issue Links

• Is contained by: SPARK-8418 Add single- and multi-value support to ML Transformers
• is related to: SPARK-30939 StringIndexer setOutputCols does not set output cols
• is required by: SPARK-22798 Add multiple column support to PySpark StringIndexer
• links to: [Github] Pull Request #9183 (yanboliang)
• links to: [Github] Pull Request #19621 (WeichenXu123)
• links to: [Github] Pull Request #20146 (viirya)
• links to: GitHub Pull Request #20146
• links to: GitHub Pull Request #25428