Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
In https://issues.apache.org/jira/browse/HIVE-964, I have a general idea about how to deal with skew join key , the key point is that use mapjoin to deal with skew key, but there has a case which troubles me:
if the two join tables have the same big skew key on one value :
for example , select * from table A join B on A.id=b.id, both table A and B have a lot of keys on id=1, in this case , if we use map join to deal with the skew key id=1 ,maybe it will OOM.
so ,how to fix this case? Will it rollback to common join ? Thanks.