Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3667

Random Assertion Error while planning

    XMLWordPrintableJSON

    Details

      Description

      git.commit.id.abbrev=b55e232

      When assertions are enabled, the below query produces an Assertion Error randomly.

      explain plan for select l_orderkey, l_partkey, l_quantity, l_shipdate, l_shipinstruct, `year`, `month` from hive.lineitem_text_partitioned_hive_hier_intint where (`year` IN (negative(-1993)) and `month`=sqrt(100)) or (`year` IN (cast(abs(-1994.0) as int)) and `month`=cast('5' as int));
      

      When Assertions are disabled, we randomly get a wrong plan where we prune away things which we should not have pruned resulting in wrong results. Again this is random.

      Wrong Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          SelectionVectorRemover
      00-04            Filter(condition=[OR(AND(=($5, NEGATIVE(-1993)), =(CAST($6):DOUBLE, 1E1)), AND(=($5, 1994), =($6, CAST('5'):INTEGER NOT NULL)))])
      00-05              Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-06                Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1991/10/lineitemaj.tbl:0+106646, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1992/10/lineitemaj.tbl:0+107653, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/10/lineitemaj.tbl:0+107846, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1995/10/lineitemaj.tbl:0+107581, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1996/10/lineitemaj.tbl:0+107072, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1997/10/lineitemaj.tbl:0+1786], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1991, 10]), Partition(values:[1992, 10]), Partition(values:[1993, 10]), Partition(values:[1994, 10]), Partition(values:[1995, 10]), Partition(values:[1996, 10]), Partition(values:[1997, 10])]]])
      

      Right Plan

      00-00    Screen
      00-01      Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-02        Project(l_orderkey=[$0], l_partkey=[$1], l_quantity=[$2], l_shipdate=[$3], l_shipinstruct=[$4], year=[$5], month=[$6])
      00-03          Project(l_orderkey=[$0], l_partkey=[$2], l_quantity=[$6], l_shipdate=[$4], l_shipinstruct=[$1], year=[$3], month=[$5])
      00-04            Scan(groupscan=[HiveScan [table=Table(dbName:default, tableName:lineitem_text_partitioned_hive_hier_intint), inputSplits=[maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1993/10/lineitemaj.tbl:0+107386, maprfs:///drill/testdata/partition_pruning/hive/text/lineitem_hierarchical_intint/1994/5/lineitemae.tbl:0+107451], columns=[`l_orderkey`, `l_partkey`, `l_quantity`, `l_shipdate`, `l_shipinstruct`, `year`, `month`], partitions= [Partition(values:[1993, 10]), Partition(values:[1994, 5])]]])
      

      I attached the data and the stack trace when we got an assertion error. Let me know if you need something.

        Attachments

        1. lineitem_hierarchical_intint.tar
          1.97 MB
          Rahul Kumar Challapalli
        2. error.log
          15 kB
          Rahul Kumar Challapalli
        3. hive.ddl
          16 kB
          Rahul Kumar Challapalli

          Issue Links

            Activity

              People

              • Assignee:
                mehant Mehant Baid
                Reporter:
                rkins Rahul Kumar Challapalli
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: