[SPARK-41511] LongToUnsafeRowMap support ignoresDuplicatedKey

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: XiDuo You
• Created: 13/Dec/22 12:11
• Updated: 15/Dec/22 13:50
• Resolved: 15/Dec/22 13:50

Description

For left semi and left anti hash join, the duplicated keys of build side have no meaning.

Previous, we supported ingore duplicated keys for UnsafeHashedRelation. We can also optimize LongHashedRelation.

Issue Links

• links to: [Github] Pull Request #39051 (ulysses-you)