[HIVE-11306] Add a bloom-1 filter for Hybrid MapJoin spills

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Hive
• Release Note: Add a bloom-1 filter to reduce Hybrid MapJoin spills
• Assignee: Wei Zheng
• Reporter: Gopal Vijayaraghavan
• Created: 18/Jul/15 06:43
• Updated: 16/Feb/16 23:52
• Resolved: 28/Oct/15 15:41

Description

HIVE-9277 implemented Spillable joins for Tez, which suffers from a corner-case performance issue when joining wide small tables against a narrow big table (like a user info table join events stream).

The fact that the wide table is spilled causes extra IO, even though the nDV of the join key might be in the thousands.

A cheap bloom-1 filter would add a massive performance gain for such queries, massively cutting down on the spill IO costs for the big-table spills.

Issue Links

• is related to: HIVE-1721 use bloom filters to improve the performance of joins
• requires: HIVE-9277 Hybrid Hybrid Grace Hash Join