Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1603

For hive tables partitioned by date, Drill scan all partitions even when there is a filter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.7.0
    • None
    • None

    Description

      git.commit.id.abbrev=6dca24a

      When we have a hive table whose partition column data type is 'DATE', Drill scans all the partitions even when we apply a filter on the partition column. This does not happen with 'INT' and 'STRING' partition datatypes

      explain plan for select l_orderkey from lineitem_text_partitioned_hive_date where dt= date '2014-10-20';
      

      Attached the output of the above query, the hive ddl, and the sample data files used

      Attachments

        1. hive.ddl
          0.8 kB
          Rahul Kumar Challapalli
        2. explain_plan.txt
          18 kB
          Rahul Kumar Challapalli
        3. lineitemaa.tbl
          1.13 MB
          Rahul Kumar Challapalli
        4. lineitemab.tbl
          1.14 MB
          Rahul Kumar Challapalli

        Activity

          People

            Unassigned Unassigned
            rkins Rahul Kumar Challapalli
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: