[IMPALA-3899] Certain join types fail to spill after they start probing if there are matches stored in the hash table

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 22/Jul/16 17:18
• Updated: 28/Sep/16 23:26
• Resolved: 28/Sep/16 23:26

Description

The fix for IMPALA-1488 disabled spilling for certain hash join types (e.g. RIGHT OUTER JOIN, FULL OUTER JOIN) if there were any matches in the hash table. The underlying cause of this (if I understand correctly) is that we can spill partitions after we've built the hash table and started probing.

As part of IMPALA-3567 I need to fix the underlying issue so that we don't spill additional build partitions while probing. As a consequence of that work, it should be possible to disable the IMPALA-1488 workaround.