[IMPALA-2715] Probe filter are not pushed to scan for joins on partitioned column

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.2
• Fix Version/s: Impala 2.5.0
• Component/s: Perf Investigation
• Labels: performance ramp-up
• Target Version: Impala 2.5.0
• Assignee: Henry Robinson
• Reporter: Mostafa Mokhtar
• Created: 25/Nov/15 23:29
• Updated: 22/Feb/16 06:16
• Resolved: 22/Feb/16 06:16

Description

Summary shows that no probe filters are pushed to the store_sales scan.

Query

select 
    count(*)
from
    store_sales,
    date_dim d1
where
    d1.d_date_sk = ss_sold_date
        and d_dom = 2
        and d1.d_quarter_name = '2000Q1'

Summary

Operator	#Hosts	Avg Time	Max Time	#Rows	Est. #Rows	Peak Mem	Est. Peak Mem	Detail
06:AGGREGATE	1	148.39us	148.39us	1	1	16.00 KB	-1 B	FINALIZE
05:EXCHANGE	1	16.63us	16.63us	1	1	0 B	-1 B	UNPARTITIONED
03:AGGREGATE	1	0ns	0ns	1	1	3.86 MB	10.00 MB
02:HASH JOIN	1	21.28s	21.28s	0	183.59K	2.02 MB	2.00 GB	INNER JOIN, BROADCAST
--04:EXCHANGE	1	22.15us	22.15us	3	-1	0 B	0 B	BROADCAST
01:SCAN HDFS	1	75.22ms	75.22ms	3	-1	1.97 MB	16.00 MB	tpcds_parquet.date_dim d1
00:SCAN HDFS	1	316.16ms	316.16ms	183.59K	183.59K	568.00 KB	16.00 MB	tpcds_parquet.store_sales_p

While for store_sales x store probe filters are pushed to the store_sales scan

select count(*)
   from  store_sales
        ,store
   where ss_store_sk = s_store_sk
   and   s_store_id="AAAAAAAABAAAAAAA";

Operator	#Hosts	Avg Time	Max Time	#Rows	Est. #Rows	Peak Mem	Est. Peak Mem	Detail
06:AGGREGATE	1	188.66us	188.66us	1	1	16.00 KB	-1 B	FINALIZE
05:EXCHANGE	1	26.11us	26.11us	1	1	0 B	-1 B	UNPARTITIONED
03:AGGREGATE	1	3.23ms	3.23ms	1	1	164.00 KB	10.00 MB
02:HASH JOIN	1	24.97ms	24.97ms	30.99K	183.59K	2.02 MB	2.00 GB	INNER JOIN, BROADCAST
--04:EXCHANGE	1	29.18us	29.18us	1	-1	0 B	0 B	BROADCAST
01:SCAN HDFS	1	3.10s	3.10s	1	-1	62.00 KB	16.00 MB	tpcds_parquet.store
00:SCAN HDFS	1	290.67ms	290.67ms	34.71K	183.59K	160.00 KB	16.00 MB	tpcds_parquet.store_sales

Issue Links

• relates to: IMPALA-2726 Templatize RawValue::GetHashValue to avoid per row PrimitiveType check for 6% Perf improvement
• relates to: IMPALA-2727 ProbeFilters can be incorrectly disabled due to cardinality agnostic check