Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3410

Partition Pruning : We are doing a prune when we shouldn't

    XMLWordPrintableJSON

Details

    Description

      git.commit.id.abbrev=60bc945

      The below plan does not look right. It should scan all the files based on the filters in the query. Also hive returned more rows than drill

      explain plan for select * from `existing_partition_pruning/lineitempart` where (dir0=1993 and columns[0] >29600) or (dir0=1994 or columns[0]>29700);
      | 00-00    Screen
      00-01      Project(*=[$0])
      00-02        Project(T70¦¦*=[$0])
      00-03          SelectionVectorRemover
      00-04            Filter(condition=[OR(AND(=($1, 1993), >(ITEM($2, 0), 29600)), =($1, 1994), >(ITEM($2, 0), 29700))])
      00-05              Project(T70¦¦*=[$0], dir0=[$1], columns=[$2])
      00-06                Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart/0_0_3.parquet], ReadEntryWithPath [path=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart/0_0_4.parquet]], selectionRoot=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart, numFiles=2, columns=[`*`]]])
       |
      

      I attached the data set used. Let me know if you need anything more

      Attachments

        1. DRILL-3410.patch
          8 kB
          Steven Phillips
        2. DRILL-3410_part2.patch
          6 kB
          Steven Phillips
        3. DRILL-3410_part2.patch
          6 kB
          Steven Phillips
        4. DRILL-3410_part2.patch
          6 kB
          Steven Phillips

        Activity

          People

            sphillips Steven Phillips
            rkins Rahul Kumar Challapalli
            Rahul Kumar Challapalli Rahul Kumar Challapalli
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: