[PIG-4130] Store/Load the same file fails for AvroStorage/OrcStorage, etc

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Fix Version/s: 0.18.0
• Component/s: impl
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 18/Aug/14 18:38
• Updated: 24/May/17 23:03

Description

The following script fail:

a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name:chararray, age:int, gpa:float);
store a into 'Avro.intermediate' using OrcStorage();
b = load 'Avro.intermediate' using OrcStorage();
c = filter b by age < 30;
store c into 'ooo';

Message:
<file Avro_1.pig, line 7, column 16> Invalid field projection. Projected field [age] does not exist.

If put a "exec" after the first store, the script success.

Pig does compile the script into two MR job, and correctly figure out the dependency of the two, but it still need to goes for "Avro.intermediate" for the schema of b when compiling, and at this time "Avro.intermediate" does not exist. This also happens to other Loaders which need to get the schema from input file, such as OrcStorage, etc.

Issue Links

• is related to: PIG-3350 JsonLoader() is not working properly for maps