Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-6

      Description

      Number of partitions scanned is 12, but expected to be 13. This was in a build with legacy aggs and joins enabled, but it's not obvious if those are related.

      FAILED:  org.apache.impala.planner.PlannerTest.testTableSample
      
      Error Message:
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         table stats: 7300 rows total
         column stats: all
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=240.27KB
         table stats: 7300 rows total
         column stats: all
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           table stats: 7300 rows total
           column stats: all
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=3650
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      where id < 10
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=239.26KB
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           predicates: id < 10
           table stats: 7300 rows total
           column stats: all
           parquet dictionary predicates: id < 10
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=365
      
      
      Stack Trace:
      java.lang.AssertionError:
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         table stats: 7300 rows total
         column stats: all
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=240.27KB
         table stats: 7300 rows total
         column stats: all
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           table stats: 7300 rows total
           column stats: all
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=3650
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      where id < 10
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=239.26KB
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           predicates: id < 10
           table stats: 7300 rows total
           column stats: all
           parquet dictionary predicates: id < 10
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=365
      
              at org.junit.Assert.fail(Assert.java:88)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:759)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:721)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:716)
              at org.apache.impala.planner.PlannerTest.testTableSample(PlannerTest.java:414)
      

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        commit 25290a00caf28cca42f74848ca0fbeaeff03489b
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Wed May 24 18:06:44 2017 -0700

        IMPALA-5358: Fix repeatable table sample.

        The bug was a simple oversight where we iterated over
        an unordered list of partitions.

        Change-Id: I9110751b075430b068b52d7441e5845f86d1b6af
        Reviewed-on: http://gerrit.cloudera.org:8080/6985
        Reviewed-by: Henry Robinson <henry@cloudera.com>
        Tested-by: Impala Public Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit 25290a00caf28cca42f74848ca0fbeaeff03489b Author: Alex Behm <alex.behm@cloudera.com> Date: Wed May 24 18:06:44 2017 -0700 IMPALA-5358 : Fix repeatable table sample. The bug was a simple oversight where we iterated over an unordered list of partitions. Change-Id: I9110751b075430b068b52d7441e5845f86d1b6af Reviewed-on: http://gerrit.cloudera.org:8080/6985 Reviewed-by: Henry Robinson <henry@cloudera.com> Tested-by: Impala Public Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            henryr Henry Robinson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development