Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6489

Fix filter push down for Hbase & Mapr-DB binary tables when convert function is used in a view

    Details

      Description

      Query

      select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as d 
      from hbase.`t` 
      where convert_from(byte_substr(row_key, 1, 8), date_epoch_be') = date '2015-06-13';
      

      returns plan with the push down:

      00-00    Screen
      00-01      Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
      00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=TestTableCompositeDate, startRow=\x00\x00\x01M\xEA7D\x00, stopRow=\x00\x00\x01M\xEF]\xA0\x00, filter=null], columns=[`row_key`]]])
      

      While the same query in a view does not:

      create view dfs.tmp.v as select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as d from hbase.`t`;
      select d from dfs.tmp.v where d = date '2015-06-13';
      
      00-00    Screen
      00-01      Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
      00-02        SelectionVectorRemover
      00-03          Filter(condition=[=(CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8)), 2015-06-13)])
      00-04            Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=TestTableCompositeDate, startRow=null, stopRow=null, filter=null], columns=[`row_key`]]])
      

      The problem that CompareFunctionsProcessor waits for ConvertExpression but receives FunctionCall with convert function. If convert function first appears in filter it is re-presented as ConvertExpression (case without view). If convert function first appears in select is re-presented as FunctionCall for convert function (case with view). The reason of such difference is the appliance of the PreProcessLogicalRel visitor. The solution in this case would be to check FunctionCall in CompareFunctionsProcessor}} and if this function call for convert from function, process it as ConvertExpression.

      https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java#L171

      https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L667

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                arina Arina Ielchiieva
                Reporter:
                arina Arina Ielchiieva
                Reviewer:
                Volodymyr Vysotskyi
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: