[SPARK-32286] Coalesce bucketed tables for shuffled hash join if applicable

• Type: Sub-task
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Cheng Su
• Reporter: Cheng Su
• Created: 12/Jul/20 22:55
• Updated: 27/Jul/20 20:11
• Resolved: 21/Jul/20 23:57

Description

Based on a follow up comment in PR #28123, where we can coalesce buckets for shuffled hash join as well. The note here is we only coalesce the buckets from shuffled hash join stream side (i.e. the side not building hash map), so we don't need to worry about OOM when coalescing multiple buckets in one task for building hash map.

Issue Links

• links to: [Github] Pull Request #29079 (c21)