Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6342

Parquet filter pushdown doesn't work in case of filtering fields inside arrays of complex fields

VotersStop watchingWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.14.0
    • 1.14.0
    • None

    Description

      Data:
      Complex_data data set is attached

      Query:

      explain plan for select * from dfs.tmp.`Complex_data` t where t.list_of_complex_fields[2].nested_field is true
      

      Expected result:
      numFiles=2
      Statistics of the file that should't be scanned:

      list_of_complex_fields:        
      .nested_field:                   BOOLEAN UNCOMPRESSED DO:0 FPO:497 SZ:41/41/1.00 VC:3 ENC:PLAIN,RLE ST:[min: false, max: false, num_nulls: 0]
      

      Actual result:
      numFiles=3
      I.e, filter pushdown is not work

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            arina Arina Ielchiieva
            angozhiy Anton Gozhiy
            Parth Chandra Parth Chandra
            Votes:
            0 Vote for this issue
            Watchers:
            3 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment