[HIVE-11401] Predicate push down does not work with Parquet when partitions are in the expression

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 2.0.0
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 29/Jul/15 17:49
• Updated: 16/Feb/16 23:51
• Resolved: 31/Jul/15 14:50

Description

When filtering Parquet tables using a partition column, the query fails saying the column does not exist:

hive> create table part1 (id int, content string) partitioned by (p string) stored as parquet;
hive> alter table part1 add partition (p='p1');
hive> insert into table part1 partition (p='p1') values (1, 'a'), (2, 'b');
hive> select id from part1 where p='p1';
Failed with exception java.io.IOException:java.lang.IllegalArgumentException: Column [p] was not found in schema!
Time taken: 0.151 seconds

It is correct that the partition column is not part of the Parquet schema. So, the fix should be to remove such expression from the Parquet PPD.

Issue Links

• links to: Review Board