[HIVE-1721] use bloom filters to improve the performance of joins

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Query Processor
• Labels: gsoc gsoc2012 optimization
• Assignee: Unassigned
• Reporter: Namit Jain
• Created: 16/Oct/10 00:53
• Updated: 18/Jul/15 06:46

Description

In case of map-joins, it is likely that the big table will not find many matching rows from the small table.
Currently, we perform a hash-map lookup for every row in the big table, which can be pretty expensive.

It might be useful to try out a bloom-filter containing all the elements in the small table.
Each element from the big table is first searched in the bloom filter, and only in case of a positive match,
the small table hash table is explored.

Issue Links

• relates to: HIVE-11306 Add a bloom-1 filter for Hybrid MapJoin spills