[PIG-3594] Pig 0.11 with HCatLoader partition filter does not push down valid conditions

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.11.1
• Fix Version/s: 0.11.2
• Assignee: Unassigned
• Reporter: Mona Chitnis
• Created: 27/Nov/13 22:49
• Updated: 11/Feb/15 09:31

Description

When running the following pig query (script.pig) :

A = LOAD ' my_data' USING org.apache.hcatalog.pig.HCatLoader();
B = FILTER A by part1=='$val1' and part2 matches '$val2';
C = FILTER B by map#'key' matches '(val3|val4)';
DUMP C;

> pig -useHCatalog -p val1=2013_01_01 val2=XYZ script.pig

This results in Pig scanning all partitions inspite of filter condition. part1=='$val1' should have been pushed to HCatLoader even though there are other conditions that cannot be pushed.

Issue Links

• duplicates: PIG-3461 Rewrite PartitionFilterOptimizer to make it work for all the cases