[PIG-1063] Pig does not call checkOutSpecs() on OutputFormat provided by StoreFunc in the multistore case

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.6.0
• Hadoop Flags: Reviewed
• Assignee: Pradeep Kamath
• Reporter: Pradeep Kamath
• Created: 29/Oct/09 21:54
• Updated: 24/Mar/10 22:15
• Resolved: 30/Oct/09 20:41

Description

A StoreFunc implementation can inform pig of an OutputFormat it uses through the getStoragePreparationClass() method. In a query with multiple stores which gets optimized into a single mapred job, Pig does not call the checkOutputSpecs() method on the outputformat. An example of such a script is:

a = load 'input.txt';
b = filter a by $0 < 10;
store b into 'output1' using StoreWithOutputFormat();
c = group a by $0;
d = foreach c generate group, COUNT(a.$0);
store d into 'output2' using StoreWithOutputFormat();