Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5358

Off-by-one error in testTableSample

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-6

      Description

      Number of partitions scanned is 12, but expected to be 13. This was in a build with legacy aggs and joins enabled, but it's not obvious if those are related.

      FAILED:  org.apache.impala.planner.PlannerTest.testTableSample
      
      Error Message:
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         table stats: 7300 rows total
         column stats: all
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=240.27KB
         table stats: 7300 rows total
         column stats: all
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           table stats: 7300 rows total
           column stats: all
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=3650
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      where id < 10
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=239.26KB
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           predicates: id < 10
           table stats: 7300 rows total
           column stats: all
           parquet dictionary predicates: id < 10
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=365
      
      
      Stack Trace:
      java.lang.AssertionError:
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         table stats: 7300 rows total
         column stats: all
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=240.27KB
         table stats: 7300 rows total
         column stats: all
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=3650
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           table stats: 7300 rows total
           column stats: all
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=3650
      
      Section PLAN of query:
      select * from functional.alltypes tablesample system(50) repeatable(1234)
      where id < 10
      
      Actual does not match expected result:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=13/24 files=13 size=258.44KB
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=96.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Expected:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
      PLAN-ROOT SINK
      |  mem-estimate=0B mem-reservation=0B
      |
      00:SCAN HDFS [functional.alltypes]
         partitions=12/24 files=12 size=239.26KB
         predicates: id < 10
         table stats: 7300 rows total
         column stats: all
         parquet dictionary predicates: id < 10
         mem-estimate=80.00MB mem-reservation=0B
         tuple-ids=0 row-size=97B cardinality=365
      
      Verbose plan:
      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
        PLAN-ROOT SINK
        |  mem-estimate=0B mem-reservation=0B
        |
        00:SCAN HDFS [functional.alltypes]
           partitions=13/24 files=13 size=258.44KB
           predicates: id < 10
           table stats: 7300 rows total
           column stats: all
           parquet dictionary predicates: id < 10
           mem-estimate=96.00MB mem-reservation=0B
           tuple-ids=0 row-size=97B cardinality=365
      
              at org.junit.Assert.fail(Assert.java:88)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:759)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:721)
              at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:716)
              at org.apache.impala.planner.PlannerTest.testTableSample(PlannerTest.java:414)
      

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              henryr Henry Robinson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: