Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3667

Random Assertion Error while planning

    XMLWordPrintableJSON

Details

    Description

      git.commit.id.abbrev=b55e232

      When assertions are enabled, the below query produces an Assertion Error randomly.

      explain plan for select l_orderkey, l_partkey, l_quantity, l_shipdate, l_shipinstruct, `year`, `month` from hive.lineitem_text_partitioned_hive_hier_intint where (`year` IN (negative(-1993)) and `month`=sqrt(100)) or (`year` IN (cast(abs(-1994.0) as int)) and `month`=cast('5' as int));
      

      When Assertions are disabled, we randomly get a wrong plan where we prune away things which we should not have pruned resulting in wrong results. Again this is random.

      Wrong Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          SelectionVectorRemover
      00-04            Filter(condition=[OR(AND(=($5, NEGATIVE(-1993)), =(CAST($6):DOUBLE, 1E1)), AND(=($5, 1994), =($6, CAST('5'):INTEGER NOT NULL)))])
      00-05              Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-06                Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1991/10/lineitemaj.tbl:0+106646, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1992/10/lineitemaj.tbl:0+107653, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/10/lineitemaj.tbl:0+107846, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1995/10/lineitemaj.tbl:0+107581, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1996/10/lineitemaj.tbl:0+107072, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1997/10/lineitemaj.tbl:0+1786], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1991, 10]), Partition(values:[1992, 10]), Partition(values:[1993, 10]), Partition(values:[1994, 10]), Partition(values:[1995, 10]), Partition(values:[1996, 10]), Partition(values:[1997, 10])]]])
      

      Right Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-04            Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/5/lineitemae.tbl:0+107451], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1993, 10]), Partition(values:[1994, 5])]]])
      

      I attached the data and the stack trace when we got an assertion error. Let me know if you need something.

      Attachments

        1. error.log
          15 kB
          Rahul Kumar Challapalli
        2. hive.ddl
          16 kB
          Rahul Kumar Challapalli
        3. lineitem_hierarchical_intint.tar
          1.97 MB
          Rahul Kumar Challapalli

        Issue Links

          Activity

            People

              mehant Mehant Baid
              rkins Rahul Kumar Challapalli
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: