[SPARK-18736] CreateMap allows non-unique keys

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.0.0
• Component/s: SQL
• Labels: bulk-closed map sql types
• Assignee: Unassigned
• Reporter: Eyal Farago
• Shepherd: Herman van Hövell
• Created: 06/Dec/16 07:01
• Updated: 21/May/19 04:13
• Resolved: 21/May/19 04:13

Description

Spark-Sql, CreateMap does not enforce unique keys, i.e. it's possible to create a map with two identical keys:

CreateMap(Literal(1), Literal(11), Literal(1), Literal(12))

This does not behave like standard maps in common programming languages.
proper behavior should be chosen:

1. first 'wins'
2. last 'wins'
3. runtime error.

GetMapValue currently implements option #1. Even if this is the desired behavior CreateMap should return a unique map.