Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3667

Random Assertion Error while planning

    Details

      Description

      git.commit.id.abbrev=b55e232

      When assertions are enabled, the below query produces an Assertion Error randomly.

      explain plan for select l_orderkey, l_partkey, l_quantity, l_shipdate, l_shipinstruct, `year`, `month` from hive.lineitem_text_partitioned_hive_hier_intint where (`year` IN (negative(-1993)) and `month`=sqrt(100)) or (`year` IN (cast(abs(-1994.0) as int)) and `month`=cast('5' as int));
      

      When Assertions are disabled, we randomly get a wrong plan where we prune away things which we should not have pruned resulting in wrong results. Again this is random.

      Wrong Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          SelectionVectorRemover
      00-04            Filter(condition=[OR(AND(=($5, NEGATIVE(-1993)), =(CAST($6):DOUBLE, 1E1)), AND(=($5, 1994), =($6, CAST('5'):INTEGER NOT NULL)))])
      00-05              Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-06                Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1991/10/lineitemaj.tbl:0+106646, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1992/10/lineitemaj.tbl:0+107653, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/10/lineitemaj.tbl:0+107846, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1995/10/lineitemaj.tbl:0+107581, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1996/10/lineitemaj.tbl:0+107072, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1997/10/lineitemaj.tbl:0+1786], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1991, 10]), Partition(values:[1992, 10]), Partition(values:[1993, 10]), Partition(values:[1994, 10]), Partition(values:[1995, 10]), Partition(values:[1996, 10]), Partition(values:[1997, 10])]]])
      

      Right Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-04            Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/5/lineitemae.tbl:0+107451], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1993, 10]), Partition(values:[1994, 5])]]])
      

      I attached the data and the stack trace when we got an assertion error. Let me know if you need something.

        Attachments

        1. hive.ddl
          16 kB
          Rahul Challapalli
        2. error.log
          15 kB
          Rahul Challapalli
        3. lineitem_hierarchical_intint.tar
          1.97 MB
          Rahul Challapalli

          Issue Links

            Activity

              People

              • Assignee:
                mehant Mehant Baid
                Reporter:
                rkins Rahul Challapalli
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: