[SPARK-43593] Support the minimum number of range shuffle partitions

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.1
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Wan Kun
• Created: 19/May/23 14:28
• Updated: 19/May/23 14:28

Description

If there are few distinct values in the RangePartitioner, there will be very few partitions that could be very large. We can append a random expression to increase the number of partitions.