Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1473

Incorrect cardinality displayed in exec summary for merging exchange on coordinator.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Impala 2.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      Consider the following exec summary for TPCH-Q21.

      select
        s_name,
        count(*) as numwait
      from
        supplier,
        lineitem l1,
        orders,
        nation
      where
        s_suppkey = l1.l_suppkey
        and o_orderkey = l1.l_orderkey
        and o_orderstatus = 'F'
        and l1.l_receiptdate > l1.l_commitdate
        and exists (
          select
            *
          from
            lineitem l2
          where
            l2.l_orderkey = l1.l_orderkey
            and l2.l_suppkey <> l1.l_suppkey
        )
        and not exists (
          select
            *
          from
            lineitem l3
          where
            l3.l_orderkey = l1.l_orderkey
            and l3.l_suppkey <> l1.l_suppkey
            and l3.l_receiptdate > l3.l_commitdate
        )
        and s_nationkey = n_nationkey
        and n_name = 'SAUDI ARABIA'
      group by
        s_name
      order by
        numwait desc,
        s_name
      limit 100
      

      Node 21 returns 100 rows, but 300 is incorrectly displayed.

      I'm not sure if this bug is specific to merging exchanges or exchanges with limits in general.

      21:MERGING-EXCHANGE        1  341.969us  341.969us      300         100          0        -1.00 B  UNPARTITIONED               
      12:TOP-N                   3    1.738ms    1.928ms      300         100   20.00 KB        4.10 KB                              
      20:AGGREGATE               3  141.305ms  141.559ms      411      10.39K    6.29 MB       10.00 MB  FINALIZE                    
      19:EXCHANGE                3  163.412us  178.198us    1.20K      10.39K          0              0  HASH(s_name)                
      11:AGGREGATE               3  159.478ms  212.594ms    1.20K      10.39K   11.47 MB       10.00 MB                              
      10:HASH JOIN               3    1s485ms    1s575ms    4.14K     600.12K  838.05 MB       14.27 MB  LEFT ANTI JOIN, PARTITIONED 
      |--18:EXCHANGE             3  293.828ms  308.780ms    3.79M     600.12K          0              0  HASH(l3.l_orderkey)         
      |  05:SCAN HDFS            3    2s787ms    3s233ms    3.79M     600.12K   65.66 MB      264.00 MB  tpch.lineitem l3            
      09:HASH JOIN               3    4s791ms    4s880ms   73.09K     600.12K  838.05 MB       33.58 MB  LEFT SEMI JOIN, PARTITIONED 
      |--17:EXCHANGE             3  380.799ms  406.256ms    6.00M       6.00M          0              0  HASH(l2.l_orderkey)         
      |  04:SCAN HDFS            3    2s745ms    3s046ms    6.00M       6.00M   64.73 MB      264.00 MB  tpch.lineitem l2            
      16:EXCHANGE                3    3.863ms    3.915ms   75.87K     600.12K          0              0  HASH(l1.l_orderkey)         
      08:HASH JOIN               3  387.206ms  400.723ms   75.87K     600.12K    8.59 MB        28.00 B  INNER JOIN, BROADCAST       
      |--15:EXCHANGE             3   15.843us   19.148us        1           1          0              0  BROADCAST                   
      |  03:SCAN HDFS            1  443.178ms  443.178ms        1           1   41.00 KB       32.00 MB  tpch.nation                 
      07:HASH JOIN               3    3s889ms    4s269ms    1.83M     600.12K  566.03 MB       13.11 MB  INNER JOIN, BROADCAST       
      |--14:EXCHANGE             3   141.48ms  144.758ms  729.41K     500.00K          0              0  BROADCAST                   
      |  02:SCAN HDFS            2    1s744ms    2s386ms  729.41K     500.00K   32.16 MB      176.00 MB  tpch.orders                 
      06:HASH JOIN               3    1s763ms    1s879ms    3.79M     600.12K   12.43 MB      472.66 KB  INNER JOIN, BROADCAST       
      |--13:EXCHANGE             3    3.112ms    5.549ms   10.00K      10.00K          0              0  BROADCAST                   
      |  00:SCAN HDFS            1  508.179ms  508.179ms   10.00K      10.00K    2.24 MB       32.00 MB  tpch.supplier               
      01:SCAN HDFS               3    1s854ms    1s895ms    3.79M     600.12K   65.38 MB      264.00 MB  tpch.lineitem l1            
      

        Attachments

          Activity

            People

            • Assignee:
              twmarshall Thomas Tauber-Marshall
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: