Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3410

Partition Pruning : We are doing a prune when we shouldn't

    Details

      Description

      git.commit.id.abbrev=60bc945

      The below plan does not look right. It should scan all the files based on the filters in the query. Also hive returned more rows than drill

      explain plan for select * from `existing_partition_pruning/lineitempart` where (dir0=1993 and columns[0] >29600) or (dir0=1994 or columns[0]>29700);
      | 00-00    Screen
      00-01      Project(*=[$0])
      00-02        Project(T70¦¦*=[$0])
      00-03          SelectionVectorRemover
      00-04            Filter(condition=[OR(AND(=($1, 1993), >(ITEM($2, 0), 29600)), =($1, 1994), >(ITEM($2, 0), 29700))])
      00-05              Project(T70¦¦*=[$0], dir0=[$1], columns=[$2])
      00-06                Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart/0_0_3.parquet], ReadEntryWithPath [path=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart/0_0_4.parquet]], selectionRoot=/drill/testdata/ctas_auto_partition/existing_partition_pruning/lineitempart, numFiles=2, columns=[`*`]]])
       |
      

      I attached the data set used. Let me know if you need anything more

        Attachments

        1. DRILL-3410_part2.patch
          6 kB
          Steven Phillips
        2. DRILL-3410_part2.patch
          6 kB
          Steven Phillips
        3. DRILL-3410_part2.patch
          6 kB
          Steven Phillips
        4. DRILL-3410.patch
          8 kB
          Steven Phillips

          Activity

            People

            • Assignee:
              sphillips Steven Phillips
              Reporter:
              rkins Rahul Challapalli
              Reviewer:
              Rahul Challapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: