[SPARK-32464] Support skew handling on join that has one side with no query stage

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Apache Spark
• Reporter: Wang, Gang
• Created: 28/Jul/20 01:22
• Updated: 28/Jul/20 01:51

Description

In our production environment, there are many bucket tables, which are used to join with other tables. And there are some skewed joins now and then. While, in current implementation, the skew join handling can only applied when both sides of a SMJ are QueryStages. So skew join handling is not able to deal with such cases.

Issue Links

• links to: [Github] Pull Request #29266 (wangshisan)