[DRILL-3667] Random Assertion Error while planning

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0
• Component/s: Query Planning & Optimization
• Assignee: Mehant Baid
• Reporter: Rahul Kumar Challapalli
• Created: 18/Aug/15 22:10
• Updated: 01/Oct/15 20:57
• Resolved: 25/Aug/15 18:33

Description

git.commit.id.abbrev=b55e232

When assertions are enabled, the below query produces an Assertion Error randomly.

explain plan for select l_orderkey, l_partkey, l_quantity, l_shipdate, l_shipinstruct, `year`, `month` from hive.lineitem_text_partitioned_hive_hier_intint where (`year` IN (negative(-1993)) and `month`=sqrt(100)) or (`year` IN (cast(abs(-1994.0) as int)) and `month`=cast('5' as int));

When Assertions are disabled, we randomly get a wrong plan where we prune away things which we should not have pruned resulting in wrong results. Again this is random.

Wrong Plan

00-00    Screen
00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
00-03          SelectionVectorRemover
00-04            Filter(condition=[OR(AND(=($5, NEGATIVE(-1993)), =(CAST($6):DOUBLE, 1E1)), AND(=($5, 1994), =($6, CAST('5'):INTEGER NOT NULL)))])
00-05              Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
00-06                Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1991/10/lineitemaj.tbl:0+106646, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1992/10/lineitemaj.tbl:0+107653, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/10/lineitemaj.tbl:0+107846, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1995/10/lineitemaj.tbl:0+107581, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1996/10/lineitemaj.tbl:0+107072, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1997/10/lineitemaj.tbl:0+1786], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1991, 10]), Partition(values:[1992, 10]), Partition(values:[1993, 10]), Partition(values:[1994, 10]), Partition(values:[1995, 10]), Partition(values:[1996, 10]), Partition(values:[1997, 10])]]])

Right Plan

00-00    Screen
00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
00-03          Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
00-04            Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/5/lineitemae.tbl:0+107451], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1993, 10]), Partition(values:[1994, 5])]]])

I attached the data and the stack trace when we got an assertion error. Let me know if you need something.

Issue Links

• is duplicated by: DRILL-3690 Partitioning pruning produces wrong results when there are nested expressions in the filter