[IMPALA-5789] Prune all partitions if an always-false runtime filter is received

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 2.11.0
• Component/s: Backend
• Labels: performance ramp-up
• Target Version: Product Backlog
• Epic Color: ghx-label-1
• Assignee: Tianyi Wang
• Reporter: Tim Armstrong
• Created: 11/Aug/17 01:34
• Updated: 07/Feb/23 11:14
• Resolved: 23/Oct/17 21:03

Description

The scenario is as follows:

• A query joins a large partitioned table with another table on a non-partition column.
• All of the rows on the build side are filtered out, so the join has an empty build.

Currently what happens is that all partitions of the large table are scanned and the runtime filter filters all the rows out after materialization.

What would ideally happen is that the scan would notice that the runtime filter always returns false and prune all of the partitions.

Issue Links

• breaks: IMPALA-6099 DCHECK in runtime filters: "Tried to increment unknown counters group"
• breaks: IMPALA-6118 Assertion failure in mem-tracker when releasing runtime filter memory
• breaks: IMPALA-6164 test_always_false_filter failure on ASAN
• is related to: IMPALA-6099 DCHECK in runtime filters: "Tried to increment unknown counters group"
• relates to: IMPALA-3430 Runtime filter : Extend runtime filter to support Min/Max values for HDFS scans
• relates to: IMPALA-3987 Short-circuit evaluation of hash joins with empty build side