Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3892

Metadata cache not being leveraged when partition pruning is taking place

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0
    • Component/s: Metadata
    • Labels:
      None

      Description

      git.commit.id.abbrev=92638dc

      As we can see from the below plan, metadata cache is not being leveraged even when the cache file is being present

      0: jdbc:drill:zk=10.10.100.190:5181> refresh table metadata dfs.`/drill/testdata/metadata_caching/lineitem_deletecache`;
      +-------+-------------------------------------------------------------------------------------------------+
      |  ok   |                                             summary                                             |
      +-------+-------------------------------------------------------------------------------------------------+
      | true  | Successfully updated metadata for table /drill/testdata/metadata_caching/lineitem_deletecache.  |
      +-------+-------------------------------------------------------------------------------------------------+
      1 row selected (0.402 seconds)
      0: jdbc:drill:zk=10.10.100.190:5181> explain plan for select count(*) from dfs.`/drill/testdata/metadata_caching/lineitem_deletecache` where dir0=2006 group by l_linestatus;
      +------+------+
      | text | json |
      +------+------+
      | 00-00    Screen
      00-01      Project(EXPR$0=[$1])
      00-02        HashAgg(group=[{0}], EXPR$0=[COUNT()])
      00-03          Project(l_linestatus=[$0])
      00-04            Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=maprfs:/drill/testdata/metadata_caching/lineitem_deletecache/2006/1/lineitem_999.parquet]], selectionRoot=maprfs:/drill/testdata/metadata_caching/lineitem_deletecache, numFiles=1, usedMetadataFile=false, columns=[`l_linestatus`, `dir0`]]])
      

      I attached the data set used. Let me know if you need anything more

        Attachments

          Activity

            People

            • Assignee:
              amansinha100 Aman Sinha
              Reporter:
              rkins Rahul Challapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: