[IMPALA-8702] PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.3.0
• Fix Version/s: Impala 3.3.0
• Component/s: Frontend
• Labels: broken-build
• Epic Color: ghx-label-7
• Assignee: Fang-Yu Rao
• Reporter: Bikramjeet Vig
• Created: 24/Jun/19 18:04
• Updated: 11/Jul/19 22:50
• Resolved: 11/Jul/19 22:46

Description

The following test case is failing in PlannerTest.testJoins.

 

Section PLAN of query:
select * from functional_text_lzo.emptytable a inner join
functional_text_lzo.alltypes b on a.f2 = b.int_col

Actual does not match expected result:
PLAN-ROOT SINK
|
02:HASH JOIN [INNER JOIN]
|  hash predicates: b.int_col = a.f2
|  runtime filters: RF000 <- a.f2
|  row-size=96B cardinality=5.66K
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|--00:SCAN HDFS [functional_text_lzo.emptytable a]
|     partitions=0/0 files=0 size=0B
|     row-size=16B cardinality=0
|
01:SCAN HDFS [functional_text_lzo.alltypes b]
   HDFS partitions=24/24 files=24 size=123.54KB
   runtime filters: RF000 -> b.int_col
   row-size=80B cardinality=5.66K

Expected:
PLAN-ROOT SINK
|
02:HASH JOIN [INNER JOIN]
|  hash predicates: b.int_col = a.f2
|  runtime filters: RF000 <- a.f2
|  row-size=96B cardinality=5.65K
|
|--00:SCAN HDFS [functional_text_lzo.emptytable a]
|     partitions=0/0 files=0 size=0B
|     row-size=16B cardinality=0
|
01:SCAN HDFS [functional_text_lzo.alltypes b]
   HDFS partitions=24/24 files=24 size=123.32KB
   runtime filters: RF000 -> b.int_col
   row-size=80B cardinality=5.65K

Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB thread-reservation=2 runtime-filters-memory=1.00MB
  PLAN-ROOT SINK
  |  mem-estimate=0B mem-reservation=0B thread-reservation=0
  |
  02:HASH JOIN [INNER JOIN]
  |  hash predicates: b.int_col = a.f2
  |  fk/pk conjuncts: assumed fk/pk
  |  runtime filters: RF000[bloom] <- a.f2
  |  mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
  |  tuple-ids=1,0 row-size=96B cardinality=5.66K
  |  in pipelines: 01(GETNEXT), 00(OPEN)
  |
  |--00:SCAN HDFS [functional_text_lzo.emptytable a]
  |     partitions=0/0 files=0 size=0B
  |     stored statistics:
  |       table: rows=unavailable size=unavailable
  |       partitions: 0/0 rows=unavailable
  |       columns missing stats: field
  |     extrapolated-rows=disabled max-scan-range-rows=0
  |     mem-estimate=0B mem-reservation=0B thread-reservation=0
  |     tuple-ids=0 row-size=16B cardinality=0
  |     in pipelines: 00(GETNEXT)
  |
  01:SCAN HDFS [functional_text_lzo.alltypes b]
     HDFS partitions=24/24 files=24 size=123.54KB
     runtime filters: RF000[bloom] -> b.int_col
     stored statistics:
       table: rows=unavailable size=unavailable
       partitions: 0/24 rows=unavailable
       columns missing stats: int_col, id, bool_col, tinyint_col, smallint_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col
     extrapolated-rows=disabled max-scan-range-rows=unavailable
     mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
     tuple-ids=1 row-size=80B cardinality=5.66K
     in pipelines: 01(GETNEXT)