[ASTERIXDB-2577] Not keeping one frame for each spilled partition before starting the probe phase in hybrid hash join

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.4.1
• Component/s: RT - Runtime
• Assignee: Unassigned
• Reporter: Shiva Jahangiri
• Created: 26/May/19 02:04
• Updated: 08/Aug/19 04:07

Description

In probe() method in optimized hybrid hash join, if insertion fails on the current spilled partition, we try to find the biggest spilled partition and flush it as a victim. If we could not find any spilled partition with size > 0, then we ASSUME that the record is large and flush it as a big object. By running customerOrderCIDHybridHashJoin_Case3() test in 

TPCHCustomerOrderHashJoinTest, it can be seen that the record size is 206 bytes (so it is smaller than a frame), but neither the spilled partitions nor the buffer manager has any frame (This is the problem, there should be 1 frame for each spilled partition). In this case, we flush the record as a large object. This means that every single record that is supposed to get inserted to a spilled partition during the probe, will get flushed separately.