[HAWQ-1597] Implement Runtime Filter for Hash Join

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0.0
• Component/s: Query Execution
• Assignee: Wen Lin
• Reporter: Wen Lin
• Created: 14/Mar/18 15:11
• Updated: 14/Sep/18 09:59
• Resolved: 20/Jul/18 10:02

Description

Bloom filter is a space-efficient probabilistic data structure invented in 1970, which is used to test whether an element is a member of a set.
Nowdays, bloom filter is widely used in OLAP or data-intensive applications to quickly filter data. It is usually implemented in OLAP systems for hash join. The basic idea is, when hash join two tables, during the build phase, build a bloomfilter information for the inner table, then push down this bloomfilter information to the scan of the outer table, so that, less tuples from the outer table will be returned to hash join node and joined with hash table. It can greatly improment the hash join performance if the selectivity is high.

Sub-Tasks

• Add A New GUC hawq_hashjoin_bloomfilter: Resolved
• Implement Deciding to Create Bloom Filter During Query Plan And Create Bloom filter For Inner Table: Resolved
• Implement Applying Bloom filter During Scan outer table: Resolved
• Implement Printing Runtime Filter Information For "explain analyze": Resolved