Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6342

Parquet filter pushdown doesn't work in case of filtering fields inside arrays of complex fields

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.14.0
    • Fix Version/s: 1.14.0
    • Component/s: None
    • Labels:

      Description

      Data:
      Complex_data data set is attached

      Query:

      explain plan for select * from dfs.tmp.`Complex_data` t where t.list_of_complex_fields[2].nested_field is true
      

      Expected result:
      numFiles=2
      Statistics of the file that should't be scanned:

      list_of_complex_fields:        
      .nested_field:                   BOOLEAN UNCOMPRESSED DO:0 FPO:497 SZ:41/41/1.00 VC:3 ENC:PLAIN,RLE ST:[min: false, max: false, num_nulls: 0]
      

      Actual result:
      numFiles=3
      I.e, filter pushdown is not work

        Attachments

          Activity

            People

            • Assignee:
              arina Arina Ielchiieva
              Reporter:
              angozhiy Anton Gozhiy
              Reviewer:
              Parth Chandra

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment