Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3702

PartitionPruning hit ClassCastException in Interpreter when the pruning filter expression is of non-nullable type.

    Details

      Description

      I have the following parquet table, created using partition by clause:

      create table mypart (id, name) partition by (id) as select cast(n_regionkey as varchar(20)), n_name from cp.`tpch/nation.parquet`;
      

      The generated parquet table consists of 5 files, each representing a partition:

      0_0_1.parquet 0_0_2.parquet 0_0_3.parquet 0_0_4.parquet 0_0_5.parquet
      

      For the following query, partition pruning works as expected:

      select id, name from mypart where id  = '0' ;
      
      00-01      Project(id=[$1], name=[$0])
      00-02        Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tmp/mypart/0_0_1.parquet]], selectionRoot=file:/tmp/mypart, numFiles=1, columns=[`id`, `name`]]])
      
      "selectionRoot" : "file:/tmp/mypart",
          "fileSet" : [ "/tmp/mypart/0_0_1.parquet" ],
          "cost" : 5.0
      

      However, the following query would hit ClassCastException when PruneScanRule calls interpreter to evaluate the filtering condition, which happens to be non-nullable.

      select id, name from mypart where concat(id,'')  = '0' ;
      
      00-05              Project(id=[$1], name=[$0])
      00-06                Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=file:/tmp/mypart]], selectionRoot=file:/tmp/mypart, numFiles=1, columns=[`id`, `name`]]])
      
      "selectionRoot" : "file:/tmp/mypart",
          "fileSet" : [ "/tmp/mypart/0_0_1.parquet", "/tmp/mypart/0_0_4.parquet", "/tmp/mypart/0_0_5.parquet", "/tmp/mypart/0_0_2.parquet", "/tmp/mypart/0_0_3.parquet" ],
          "cost" : 25.0
        },
      

      Here is the error for the ClassCastException, raised in Interpreter:

      java.lang.ClassCastException: org.apache.drill.exec.expr.holders.BitHolder cannot be cast to org.apache.drill.exec.expr.holders.NullableBitHolder
      

      The cause of the problem is that PruneScanRule assumes the output type of a filter condition is NullableBit, while in this case the filter condition is Bit type, which leads to ClassCastException.

        Attachments

          Activity

            People

            • Assignee:
              jni Jinfeng Ni
              Reporter:
              jni Jinfeng Ni
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: