Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1465

Partition filter pushdown not happening for EasyGroupScan

Details

    Description

      For EasyGroupScan that handles Json and Text formats, we are not getting the directory partition filters pushed down into the Scan. One part of the reason is that the EasyGroupScan digest for a new scan that has a smaller selection of files is currently the same as the previous scan, so the planner ignores the new scan.

      A second related issue is that the Filter can appear above either a Scan or a Project, so we need to be able to handle both cases during pushdown.

      Attachments

        Activity

          amansinha100 Aman Sinha added a comment -

          Uploaded a patch for this issue. Also added a whole bunch of Junit tests for push partition filter on all 3 formats: parquet, json and csv.

          amansinha100 Aman Sinha added a comment - Uploaded a patch for this issue. Also added a whole bunch of Junit tests for push partition filter on all 3 formats: parquet, json and csv.
          amansinha100 Aman Sinha added a comment -

          Reviewboard link for this fix: https://reviews.apache.org/r/26189/

          amansinha100 Aman Sinha added a comment - Reviewboard link for this fix: https://reviews.apache.org/r/26189/

          fixed in 1959ec667658b7d3cbcea57cc66341b531e00658

          sphillips Steven Phillips added a comment - fixed in 1959ec667658b7d3cbcea57cc66341b531e00658

          People

            DrillCommitter DrillCommitter
            amansinha100 Aman Sinha
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: