Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3141

Scan nodes wait till RUNTIME_FILTER_WAIT_TIME_MS expires even if the filter is cancelled due to "FP Rate Too High"

    Details

      Description

      If runtime filter deemed not useable due to high FPP the scan node is not notified and waits till RUNTIME_FILTER_WAIT_TIME_MS expires

        HASH_JOIN_NODE (id=8):(Total: 20s586ms, non-child: 609.357ms, % non-child: 2.96%)
                ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously, Build-Side Runtime-Filter Disabled (FP Rate Too High)
                 - BuildPartitionTime: 189.995ms
                 - BuildRows: 3.85M (3850000)
                 - BuildRowsPartitioned: 3.85M (3850000)
                 - BuildTime: 348.508ms
      

      From the log

      I0304 08:43:23.084043 64653 hdfs-scan-node.cc:209] Only following filters arrived: 0, 3, 4, waited 19s253ms
      
      08:HASH JOIN [INNER JOIN, BROADCAST]
      |  hash predicates: ss_customer_sk = c_customer_sk
      |  runtime filters: RF002 <- c_customer_sk
      |  hosts=20 per-host-mem=64.62MB
      |  tuple-ids=1,2,0,3 row-size=147B cardinality=25193874
      |
      
                Filter 2:
                   - Rows processed: 196.60K (196596)
                   - Rows rejected: 0 (0)
                   - Rows total: 196.61K (196608)
      

        Attachments

        1. q19_20sec_wait.txt
          91 kB
          Mostafa Mokhtar
        2. q19_profile.txt
          550 kB
          Mostafa Mokhtar

          Activity

            People

            • Assignee:
              henryr Henry Robinson
              Reporter:
              mmokhtar Mostafa Mokhtar
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: