[IMPALA-5358] Off-by-one error in testTableSample

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Frontend
• Labels: broken-build
• Epic Color: ghx-label-6
• Assignee: Alexander Behm
• Reporter: Henry Robinson
• Created: 24/May/17 17:51
• Updated: 25/May/17 14:52
• Resolved: 25/May/17 14:52

Description

Number of partitions scanned is 12, but expected to be 13. This was in a build with legacy aggs and joins enabled, but it's not obvious if those are related.

FAILED:  org.apache.impala.planner.PlannerTest.testTableSample

Error Message:

Section PLAN of query:
select * from functional.alltypes tablesample system(50) repeatable(1234)

Actual does not match expected result:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=13/24 files=13 size=258.44KB
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   table stats: 7300 rows total
   column stats: all
   mem-estimate=96.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=3650

Expected:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=12/24 files=12 size=240.27KB
   table stats: 7300 rows total
   column stats: all
   mem-estimate=80.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=3650

Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
  PLAN-ROOT SINK
  |  mem-estimate=0B mem-reservation=0B
  |
  00:SCAN HDFS [functional.alltypes]
     partitions=13/24 files=13 size=258.44KB
     table stats: 7300 rows total
     column stats: all
     mem-estimate=96.00MB mem-reservation=0B
     tuple-ids=0 row-size=97B cardinality=3650

Section PLAN of query:
select * from functional.alltypes tablesample system(50) repeatable(1234)
where id < 10

Actual does not match expected result:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=13/24 files=13 size=258.44KB
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   predicates: id < 10
   table stats: 7300 rows total
   column stats: all
   parquet dictionary predicates: id < 10
   mem-estimate=96.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=365

Expected:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=12/24 files=12 size=239.26KB
   predicates: id < 10
   table stats: 7300 rows total
   column stats: all
   parquet dictionary predicates: id < 10
   mem-estimate=80.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=365

Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
  PLAN-ROOT SINK
  |  mem-estimate=0B mem-reservation=0B
  |
  00:SCAN HDFS [functional.alltypes]
     partitions=13/24 files=13 size=258.44KB
     predicates: id < 10
     table stats: 7300 rows total
     column stats: all
     parquet dictionary predicates: id < 10
     mem-estimate=96.00MB mem-reservation=0B
     tuple-ids=0 row-size=97B cardinality=365


Stack Trace:
java.lang.AssertionError:
Section PLAN of query:
select * from functional.alltypes tablesample system(50) repeatable(1234)

Actual does not match expected result:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=13/24 files=13 size=258.44KB
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   table stats: 7300 rows total
   column stats: all
   mem-estimate=96.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=3650

Expected:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=12/24 files=12 size=240.27KB
   table stats: 7300 rows total
   column stats: all
   mem-estimate=80.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=3650

Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
  PLAN-ROOT SINK
  |  mem-estimate=0B mem-reservation=0B
  |
  00:SCAN HDFS [functional.alltypes]
     partitions=13/24 files=13 size=258.44KB
     table stats: 7300 rows total
     column stats: all
     mem-estimate=96.00MB mem-reservation=0B
     tuple-ids=0 row-size=97B cardinality=3650

Section PLAN of query:
select * from functional.alltypes tablesample system(50) repeatable(1234)
where id < 10

Actual does not match expected result:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=13/24 files=13 size=258.44KB
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   predicates: id < 10
   table stats: 7300 rows total
   column stats: all
   parquet dictionary predicates: id < 10
   mem-estimate=96.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=365

Expected:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B
|
00:SCAN HDFS [functional.alltypes]
   partitions=12/24 files=12 size=239.26KB
   predicates: id < 10
   table stats: 7300 rows total
   column stats: all
   parquet dictionary predicates: id < 10
   mem-estimate=80.00MB mem-reservation=0B
   tuple-ids=0 row-size=97B cardinality=365

Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
  PLAN-ROOT SINK
  |  mem-estimate=0B mem-reservation=0B
  |
  00:SCAN HDFS [functional.alltypes]
     partitions=13/24 files=13 size=258.44KB
     predicates: id < 10
     table stats: 7300 rows total
     column stats: all
     parquet dictionary predicates: id < 10
     mem-estimate=96.00MB mem-reservation=0B
     tuple-ids=0 row-size=97B cardinality=365

        at org.junit.Assert.fail(Assert.java:88)
        at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:759)
        at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:721)
        at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:716)
        at org.apache.impala.planner.PlannerTest.testTableSample(PlannerTest.java:414)