[PIG-3322] AVRO: AvroStorage give NPE on reading file with union as top level schema

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.2
• Fix Version/s: 0.12.0
• Component/s: piggybank
• Labels: patch
• Tags: AvroStorage
• Assignee: Viraj Bhat
• Reporter: Egil Sorensen
• Created: 13/May/13 23:44
• Updated: 14/Oct/13 16:46
• Resolved: 04/Jun/13 01:03

Description

I am getting NPE when loading a file with AvroStorage a file that has schema like:

["null",{"type":"record","name":"TUPLE_0","fields":[{"name":"name","type":["null","string"],"doc":"autogenerated from Pig Field Schema"},{"name":"age","type":["null","int"],"doc":"autogenerated from Pig Field Schema"},{"name":"gpa","type":["null","double"],"doc":"autogenerated from Pig Field Schema"}]}]

E.g. see the e2e style test, which fails on this:

                        {
                        'num' => 4,
                        # storing file with Pig type tuple relying on conversion to record
                        # loading using stored schemas 
                        'notmq' => 1,
                        'pig' => q\
a = load ':INPATH:/singlefile/studentcomplextab10k' using PigStorage() as (m:[], t:(name:chararray, age:int, gpa:double), b:{t:(name:chararray, age:int, gpa:double)});
b = foreach a generate t;
describe b;
store b into ':OUTPATH:.intermediate' USING org.apache.pig.piggybank.storage.avro.AvroStorage();

exec;

-- Read back what was stored with Avro
u = load ':OUTPATH:.intermediate' USING org.apache.pig.piggybank.storage.avro.AvroStorage();
describe u;
store u into ':OUTPATH:';
\,
                        'verify_pig_script' => q\
a = load ':INPATH:/singlefile/studentcomplextab10k' using PigStorage() as (m:[], t:(name:chararray, age:int, gpa:double), b:{t:(name:chararray, age:int, gpa:double)});
b = foreach a generate t;
describe b;
store b into ':OUTPATH:';
\,
                        },

Issue Links

• duplicates: PIG-2330 Problem in org.apache.pig.piggybank.storage.avro.AvroStorage when storing a record with a single field.