[IMPALA-1473] Incorrect cardinality displayed in exec summary for merging exchange on coordinator.

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.0
• Fix Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: ramp-up
• Target Version: Product Backlog
• Assignee: Thomas Tauber-Marshall
• Reporter: Alexander Behm
• Created: 15/Nov/14 07:43
• Updated: 04/Jan/17 23:58
• Resolved: 17/Oct/16 16:40

Description

Consider the following exec summary for TPCH-Q21.

select
  s_name,
  count(*) as numwait
from
  supplier,
  lineitem l1,
  orders,
  nation
where
  s_suppkey = l1.l_suppkey
  and o_orderkey = l1.l_orderkey
  and o_orderstatus = 'F'
  and l1.l_receiptdate > l1.l_commitdate
  and exists (
    select
      *
    from
      lineitem l2
    where
      l2.l_orderkey = l1.l_orderkey
      and l2.l_suppkey <> l1.l_suppkey
  )
  and not exists (
    select
      *
    from
      lineitem l3
    where
      l3.l_orderkey = l1.l_orderkey
      and l3.l_suppkey <> l1.l_suppkey
      and l3.l_receiptdate > l3.l_commitdate
  )
  and s_nationkey = n_nationkey
  and n_name = 'SAUDI ARABIA'
group by
  s_name
order by
  numwait desc,
  s_name
limit 100

Node 21 returns 100 rows, but 300 is incorrectly displayed.

I'm not sure if this bug is specific to merging exchanges or exchanges with limits in general.

21:MERGING-EXCHANGE        1  341.969us  341.969us      300         100          0        -1.00 B  UNPARTITIONED               
12:TOP-N                   3    1.738ms    1.928ms      300         100   20.00 KB        4.10 KB                              
20:AGGREGATE               3  141.305ms  141.559ms      411      10.39K    6.29 MB       10.00 MB  FINALIZE                    
19:EXCHANGE                3  163.412us  178.198us    1.20K      10.39K          0              0  HASH(s_name)                
11:AGGREGATE               3  159.478ms  212.594ms    1.20K      10.39K   11.47 MB       10.00 MB                              
10:HASH JOIN               3    1s485ms    1s575ms    4.14K     600.12K  838.05 MB       14.27 MB  LEFT ANTI JOIN, PARTITIONED 
|--18:EXCHANGE             3  293.828ms  308.780ms    3.79M     600.12K          0              0  HASH(l3.l_orderkey)         
|  05:SCAN HDFS            3    2s787ms    3s233ms    3.79M     600.12K   65.66 MB      264.00 MB  tpch.lineitem l3            
09:HASH JOIN               3    4s791ms    4s880ms   73.09K     600.12K  838.05 MB       33.58 MB  LEFT SEMI JOIN, PARTITIONED 
|--17:EXCHANGE             3  380.799ms  406.256ms    6.00M       6.00M          0              0  HASH(l2.l_orderkey)         
|  04:SCAN HDFS            3    2s745ms    3s046ms    6.00M       6.00M   64.73 MB      264.00 MB  tpch.lineitem l2            
16:EXCHANGE                3    3.863ms    3.915ms   75.87K     600.12K          0              0  HASH(l1.l_orderkey)         
08:HASH JOIN               3  387.206ms  400.723ms   75.87K     600.12K    8.59 MB        28.00 B  INNER JOIN, BROADCAST       
|--15:EXCHANGE             3   15.843us   19.148us        1           1          0              0  BROADCAST                   
|  03:SCAN HDFS            1  443.178ms  443.178ms        1           1   41.00 KB       32.00 MB  tpch.nation                 
07:HASH JOIN               3    3s889ms    4s269ms    1.83M     600.12K  566.03 MB       13.11 MB  INNER JOIN, BROADCAST       
|--14:EXCHANGE             3   141.48ms  144.758ms  729.41K     500.00K          0              0  BROADCAST                   
|  02:SCAN HDFS            2    1s744ms    2s386ms  729.41K     500.00K   32.16 MB      176.00 MB  tpch.orders                 
06:HASH JOIN               3    1s763ms    1s879ms    3.79M     600.12K   12.43 MB      472.66 KB  INNER JOIN, BROADCAST       
|--13:EXCHANGE             3    3.112ms    5.549ms   10.00K      10.00K          0              0  BROADCAST                   
|  00:SCAN HDFS            1  508.179ms  508.179ms   10.00K      10.00K    2.24 MB       32.00 MB  tpch.supplier               
01:SCAN HDFS               3    1s854ms    1s895ms    3.79M     600.12K   65.38 MB      264.00 MB  tpch.lineitem l1