[PIG-2266] bug with input file joining optimization in Pig

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.10.0
• Fix Version/s: 0.12.0
• Component/s: impl
• Assignee: Joseph Adler
• Reporter: Joseph Adler
• Created: 06/Sep/11 18:27
• Updated: 14/Oct/13 16:46
• Resolved: 29/Jan/13 20:57

Description

In src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java, the function hasTooManyInputFiles instantiated a LoadFunc instance, then calls setLocation before calling setUDFContextSignature. This is inconsistent with the documentation for the LoadFunc interface (see http://pig.apache.org/docs/r0.9.0/api/org/apache/pig/LoadFunc.html#setUDFContextSignature(java.lang.String)). (We've written UDFs that assume that setUDFContextSignature is called first.)

I think you can fix this by adding

loader.setUDFContextSignature(ld.getSignature());

Before

loader.setLocation(location, job);

Issue Links

• blocks: PIG-3015 Rewrite of AvroStorage