[HIVE-8518] Compile time skew join optimization returns duplicated results

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 1.0.2, 1.1.0
• Component/s: Logical Optimizer
• Assignee: Rui Li
• Reporter: Rui Li
• Created: 20/Oct/14 08:26
• Updated: 10/Aug/15 22:33
• Resolved: 21/Oct/14 17:45

Description

Compile time skew join optimization clones the join operator tree and unions the results.
The problem here is that we don't properly insert the predicate for the cloned join (relying on an assert statement).

To reproduce the issue, run the simple query:

select * from tbl1 join tbl2 on tbl1.key=tbl2.key;

And suppose there's some skew in tbl1 (specify skew with CREATE or ALTER statement).
Duplicated results will be returned if you set hive.optimize.skewjoin.compiletime=true.

Issue Links

• is depended upon by: HIVE-8535 Enable compile time skew join optimization for spark [Spark Branch]