Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6489

Fix filter push down for Hbase & Mapr-DB binary tables when convert function is used in a view

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Query

      select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as d 
      from hbase.`t` 
      where convert_from(byte_substr(row_key, 1, 8), date_epoch_be') = date '2015-06-13';
      

      returns plan with the push down:

      00-00    Screen
      00-01      Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
      00-02        Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=TestTableCompositeDate, startRow=\x00\x00\x01M\xEA7D\x00, stopRow=\x00\x00\x01M\xEF]\xA0\x00, filter=null], columns=[`row_key`]]])
      

      While the same query in a view does not:

      create view dfs.tmp.v as select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as d from hbase.`t`;
      select d from dfs.tmp.v where d = date '2015-06-13';
      
      00-00    Screen
      00-01      Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
      00-02        SelectionVectorRemover
      00-03          Filter(condition=[=(CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8)), 2015-06-13)])
      00-04            Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=TestTableCompositeDate, startRow=null, stopRow=null, filter=null], columns=[`row_key`]]])
      

      The problem that CompareFunctionsProcessor waits for ConvertExpression but receives FunctionCall with convert function. If convert function first appears in filter it is re-presented as ConvertExpression (case without view). If convert function first appears in select is re-presented as FunctionCall for convert function (case with view). The reason of such difference is the appliance of the PreProcessLogicalRel visitor. The solution in this case would be to check FunctionCall in CompareFunctionsProcessor}} and if this function call for convert from function, process it as ConvertExpression.

      https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java#L171

      https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L667

        Attachments

          Activity

            People

            • Assignee:
              arina Arina Ielchiieva
              Reporter:
              arina Arina Ielchiieva
              Reviewer:
              Vova Vysotskyi

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment