[DATAFU-171] dedupWithCombiner to support Cols list in the order / group by params

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.1
• Fix Version/s: 1.8.0
• Assignee: Unassigned
• Reporter: Shaked Aharon
• Created: 13/Nov/22 13:34
• Updated: 27/Jun/23 06:57
• Resolved: 11/Jan/23 12:33

Description

Currently, dedupWithCombiner only support a single column as its groupBy / orderBy keys. you can still leverage multiple keys, but you will need to add them as column with some expression to unify the columns.

I would suggest adding support of multiple columns in the function itself without having to modify the df prior to that