Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8702

PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 3.3.0
    • Fix Version/s: Impala 3.3.0
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-7

      Description

      The following test case is failing in PlannerTest.testJoins.

       

      Section PLAN of query:
      select * from functional_text_lzo.emptytable a inner join
      functional_text_lzo.alltypes b on a.f2 = b.int_col
      
      Actual does not match expected result:
      PLAN-ROOT SINK
      |
      02:HASH JOIN [INNER JOIN]
      |  hash predicates: b.int_col = a.f2
      |  runtime filters: RF000 <- a.f2
      |  row-size=96B cardinality=5.66K
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      |--00:SCAN HDFS [functional_text_lzo.emptytable a]
      |     partitions=0/0 files=0 size=0B
      |     row-size=16B cardinality=0
      |
      01:SCAN HDFS [functional_text_lzo.alltypes b]
         HDFS partitions=24/24 files=24 size=123.54KB
         runtime filters: RF000 -> b.int_col
         row-size=80B cardinality=5.66K
      
      Expected:
      PLAN-ROOT SINK
      |
      02:HASH JOIN [INNER JOIN]
      |  hash predicates: b.int_col = a.f2
      |  runtime filters: RF000 <- a.f2
      |  row-size=96B cardinality=5.65K
      |
      |--00:SCAN HDFS [functional_text_lzo.emptytable a]
      |     partitions=0/0 files=0 size=0B
      |     row-size=16B cardinality=0
      |
      01:SCAN HDFS [functional_text_lzo.alltypes b]
         HDFS partitions=24/24 files=24 size=123.32KB
         runtime filters: RF000 -> b.int_col
         row-size=80B cardinality=5.65K
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB thread-reservation=2 runtime-filters-memory=1.00MB
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B thread-reservation=0
        |
        02:HASH JOIN [INNER JOIN]
        |  hash predicates: b.int_col = a.f2
        |  fk/pk conjuncts: assumed fk/pk
        |  runtime filters: RF000[bloom] <- a.f2
        |  mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
        |  tuple-ids=1,0 row-size=96B cardinality=5.66K
        |  in pipelines: 01(GETNEXT), 00(OPEN)
        |
        |--00:SCAN HDFS [functional_text_lzo.emptytable a]
        |     partitions=0/0 files=0 size=0B
        |     stored statistics:
        |       table: rows=unavailable size=unavailable
        |       partitions: 0/0 rows=unavailable
        |       columns missing stats: field
        |     extrapolated-rows=disabled max-scan-range-rows=0
        |     mem-estimate=0B mem-reservation=0B thread-reservation=0
        |     tuple-ids=0 row-size=16B cardinality=0
        |     in pipelines: 00(GETNEXT)
        |
        01:SCAN HDFS [functional_text_lzo.alltypes b]
           HDFS partitions=24/24 files=24 size=123.54KB
           runtime filters: RF000[bloom] -> b.int_col
           stored statistics:
             table: rows=unavailable size=unavailable
             partitions: 0/24 rows=unavailable
             columns missing stats: int_col, id, bool_col, tinyint_col, smallint_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col
           extrapolated-rows=disabled max-scan-range-rows=unavailable
           mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
           tuple-ids=1 row-size=80B cardinality=5.66K
           in pipelines: 01(GETNEXT)
      

        Attachments

          Activity

            People

            • Assignee:
              fangyurao Fang-Yu Rao
              Reporter:
              bikramjeet.vig Bikramjeet Vig
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: