[SPARK-30449] Introducing get_dummies method in pyspark

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.4
• Component/s: PySpark
• Labels: bulk-closed
• Flags: Important
• Assignee: Unassigned
• Reporter: Krishna Kumar Tiwari
• Created: 07/Jan/20 17:36
• Updated: 25/May/21 01:52
• Resolved: 25/May/21 01:45

Description

Introducing get_dummies method in pyspark same as pandas.

Many times when using categorical variable and we want to flatten the data to do one-hot encoding to generate columns and fill the matrix, get_dummies is very useful in that scenario.

 

The objective here is to introduce get_dummies to pyspark.