[HIVE-24016] Share bloom filter construction branch in multi column semijoin reducers

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 07/Aug/20 12:42
• Updated: 12/Aug/20 17:08

Description

In HIVE-21196, we add a transformation capable of merging single column semijoin reducers to multi column semijoin reducer.

Currently it transforms the subplan SB0 to subplan SB1.

SB0

                                      / RS -> TS_1[Editor] 
         / SEL[fname] - GB - RS - GB -  RS -> TS_0[Author] 
 SOURCE 
         \ SEL[lname] - GB - RS - GB -  RS -> TS_0[Author]
	                              \ RS -> TS_1[Editor]

TS_0[Author] - FIL[in_bloom(fname) ^ in_bloom(lname)]
TS_1[Editor] - FIL[in_bloom(fname) ^ in_bloom(lname)]  

SB1

         / SEL[fname,lname] - GB - RS - GB - RS -> TS[Author] - FIL[in_bloom(hash(fname,lname))]
 SOURCE  
         \ SEL[fname,lname] - GB - RS - GB - RS -> TS[Editor] - FIL[in_bloom(hash(fname,lname))]

Observe that in SB1 we could share the common path that creates the bloom filter (SEL - GB - RS -GB) to obtain a plan like SB2.

SB2

					   / RS -> TS[Author] - FIL[in_bloom(hash(fname,lname))]
 SOURCE - SEL[fname,lname] - GB - RS - GB -
					   \ RS -> TS[Editor] - FIL[in_bloom(hash(fname,lname))]

Issue Links

• requires: HIVE-21196 Support semijoin reduction on multiple column join