-
Type:
Sub-task
-
Status: In Progress
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: SQL
-
Labels:None
For a two way shuffle join, if one or multiple groups are skewed in one table (say left table) but having a relative small number of rows in another table (say right table), we can use broadcast join for these skewed groups and use shuffle join for other groups.
- is duplicated by
-
SPARK-16753 Spark SQL doesn't handle skewed dataset joins properly
-
- Resolved
-
-
SPARK-17788 RangePartitioner results in few very large tasks and many small to empty tasks
-
- Resolved
-
-
SPARK-24799 A solution of dealing with data skew in left,right,inner join
-
- Resolved
-
- links to