Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6342

Parquet filter pushdown doesn't work in case of filtering fields inside arrays of complex fields

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.14.0
    • Fix Version/s: 1.14.0
    • Component/s: None
    • Labels:

      Description

      Data:
      Complex_data data set is attached

      Query:

      explain plan for select * from dfs.tmp.`Complex_data` t where t.list_of_complex_fields[2].nested_field is true
      

      Expected result:
      numFiles=2
      Statistics of the file that should't be scanned:

      list_of_complex_fields:        
      .nested_field:                   BOOLEAN UNCOMPRESSED DO:0 FPO:497 SZ:41/41/1.00 VC:3 ENC:PLAIN,RLE ST:[min: false, max: false, num_nulls: 0]
      

      Actual result:
      numFiles=3
      I.e, filter pushdown is not work

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                arina Arina Ielchiieva
                Reporter:
                angozhiy Anton Gozhiy
                Reviewer:
                Parth Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: