Description
I am getting NPE when loading a file with AvroStorage a file that has schema like:
["null",{"type":"record","name":"TUPLE_0","fields":[{"name":"name","type":["null","string"],"doc":"autogenerated from Pig Field Schema"},{"name":"age","type":["null","int"],"doc":"autogenerated from Pig Field Schema"},{"name":"gpa","type":["null","double"],"doc":"autogenerated from Pig Field Schema"}]}]
E.g. see the e2e style test, which fails on this:
{ 'num' => 4, # storing file with Pig type tuple relying on conversion to record # loading using stored schemas 'notmq' => 1, 'pig' => q\ a = load ':INPATH:/singlefile/studentcomplextab10k' using PigStorage() as (m:[], t:(name:chararray, age:int, gpa:double), b:{t:(name:chararray, age:int, gpa:double)}); b = foreach a generate t; describe b; store b into ':OUTPATH:.intermediate' USING org.apache.pig.piggybank.storage.avro.AvroStorage(); exec; -- Read back what was stored with Avro u = load ':OUTPATH:.intermediate' USING org.apache.pig.piggybank.storage.avro.AvroStorage(); describe u; store u into ':OUTPATH:'; \, 'verify_pig_script' => q\ a = load ':INPATH:/singlefile/studentcomplextab10k' using PigStorage() as (m:[], t:(name:chararray, age:int, gpa:double), b:{t:(name:chararray, age:int, gpa:double)}); b = foreach a generate t; describe b; store b into ':OUTPATH:'; \, },
Attachments
Attachments
Issue Links
- duplicates
-
PIG-2330 Problem in org.apache.pig.piggybank.storage.avro.AvroStorage when storing a record with a single field.
- Open