[HIVE-7159] For inner joins push a 'is not null predicate' to the join sources for every non nullSafe join condition

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Assignee: Harish Butani
• Reporter: Harish Butani
• Created: 31/May/14 17:39
• Updated: 13/Nov/14 19:41
• Resolved: 23/Jun/14 21:09

Description

A join B on A.x = B.y
can be transformed to
(A where x is not null) join (B where y is not null) on A.x = B.y

Apart from avoiding shuffling null keyed rows it also avoids issues with reduce-side skew when there are a lot of null values in the data.

Thanks to gopalv for the analysis and coming up with the solution.

Issue Links

• supercedes: HIVE-7151 NULL keys should not be shuffled for inner equi joins