Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4201

DrillPushFilterPastProject should allow partial filter pushdown.

    XMLWordPrintableJSON

    Details

      Description

      Currently, DrillPushFilterPastProjectRule will stop pushing the filter down, if the filter itself has ITEM or FLATTEN function, or its input reference is referring to an ITEM or FLATTEN function. However, in case that the filter is a conjunction of multiple sub-filters, some of them refer to ITEM or FLATTEN but the other not, then we should allow partial filter to be pushed down. For instance,

      WHERE partition_col > 10 and flatten_output_col = 'ABC'.

      The "flatten_output_col" comes from the output of FLATTEN operator, and therefore flatten_output_col = 'ABC' should not pushed past the project. But partiion_col > 10 should be pushed down, such that we could trigger the pruning rule to apply partition pruning.

      It would be improve Drill query performance, when the partially pushed filter leads to partition pruning, or the partially pushed filter results in early filtering in upstream operator.

        Attachments

          Activity

            People

            • Assignee:
              jni Jinfeng Ni
              Reporter:
              jni Jinfeng Ni
              Reviewer:
              Victoria Markman
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: