Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7022

Partition pruning is not happening the first time after the metadata auto refresh

    XMLWordPrintableJSON

Details

    Description

      Data creation:

      1. Create table:
        create table dfs.tmp.`orders` 
        partition by (o_orderstatus)
        as select * from cp.`tpch/orders.parquet`
        
      2. Create table metadata:
        refresh table metadata dfs.tmp.`orders`
        

      Steps:

      1. Modify the table to trigger metadata auto refresh:
        hadoop fs -mkdir /tmp/orders/111
        
      2. Run the query:
        explain plan for 
        select * from dfs.tmp.`orders` 
        where o_orderstatus = 'O' and o_orderdate < '1995-03-10'
        

      Expected result:
      Partition pruning happens:

      ... numFiles=1, numRowGroups=1, usedMetadataFile=true ...
      

      Actual result:
      Partition pruning doesn't happen:

      ... numFiles=1, numRowGroups=3, usedMetadataFile=true
      

      Note: It is being reproduced only the first time after auto refresh, after repeating the query it works as expected.

      Attachments

        Issue Links

          Activity

            People

              volodymyr Vova Vysotskyi
              angozhiy Anton Gozhiy
              Vitalii Diravka Vitalii Diravka
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: