Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 2.5.0
Description
If runtime filter deemed not useable due to high FPP the scan node is not notified and waits till RUNTIME_FILTER_WAIT_TIME_MS expires
HASH_JOIN_NODE (id=8):(Total: 20s586ms, non-child: 609.357ms, % non-child: 2.96%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously, Build-Side Runtime-Filter Disabled (FP Rate Too High)
- BuildPartitionTime: 189.995ms
- BuildRows: 3.85M (3850000)
- BuildRowsPartitioned: 3.85M (3850000)
- BuildTime: 348.508ms
From the log
I0304 08:43:23.084043 64653 hdfs-scan-node.cc:209] Only following filters arrived: 0, 3, 4, waited 19s253ms
08:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: ss_customer_sk = c_customer_sk | runtime filters: RF002 <- c_customer_sk | hosts=20 per-host-mem=64.62MB | tuple-ids=1,2,0,3 row-size=147B cardinality=25193874 |
Filter 2: - Rows processed: 196.60K (196596) - Rows rejected: 0 (0) - Rows total: 196.61K (196608)