Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1603

For hive tables partitioned by date, Drill scan all partitions even when there is a filter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.7.0
    • None
    • None

    Description

      git.commit.id.abbrev=6dca24a

      When we have a hive table whose partition column data type is 'DATE', Drill scans all the partitions even when we apply a filter on the partition column. This does not happen with 'INT' and 'STRING' partition datatypes

      explain plan for select l_orderkey from lineitem_text_partitioned_hive_date where dt= date '2014-10-20';
      

      Attached the output of the above query, the hive ddl, and the sample data files used

      Attachments

        1. explain_plan.txt
          18 kB
          Rahul Kumar Challapalli
        2. hive.ddl
          0.8 kB
          Rahul Kumar Challapalli
        3. lineitemaa.tbl
          1.13 MB
          Rahul Kumar Challapalli
        4. lineitemab.tbl
          1.14 MB
          Rahul Kumar Challapalli

        Activity

          People

            Unassigned Unassigned
            rkins Rahul Kumar Challapalli
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: