Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11401

Predicate push down does not work with Parquet when partitions are in the expression

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      When filtering Parquet tables using a partition column, the query fails saying the column does not exist:

      hive> create table part1 (id int, content string) partitioned by (p string) stored as parquet;
      hive> alter table part1 add partition (p='p1');
      hive> insert into table part1 partition (p='p1') values (1, 'a'), (2, 'b');
      hive> select id from part1 where p='p1';
      Failed with exception java.io.IOException:java.lang.IllegalArgumentException: Column [p] was not found in schema!
      Time taken: 0.151 seconds
      

      It is correct that the partition column is not part of the Parquet schema. So, the fix should be to remove such expression from the Parquet PPD.

        Attachments

        1. HIVE-11401.1.patch
          27 kB
          Sergio Peña
        2. HIVE-11401.2.patch
          27 kB
          Sergio Peña

          Issue Links

            Activity

              People

              • Assignee:
                spena Sergio Peña
                Reporter:
                spena Sergio Peña
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: