Description
Piggybank - AvroStorage. When merging multiple schemas where default values have been specified in the avro schema;
The AvroStorage puts nulls in the merged data set.
==> Employee3.avro <==
{
"type" : "record",
"name" : "employee",
"fields":[
,
,
] }
==> Employee4.avro <==
{
"type" : "record",
"name" : "employee",
"fields":[
,
,
,
] }
==> Employee6.avro <==
{
"type" : "record",
"name" : "employee",
"fields":[
,
,
,
,
,
] }
The pig script:
employee = load 'employee
.ser' using org.apache.pig.piggybank.storage.avro.AvroStorage('multiple_schemas');
describe employee;
dump employee;
Output Schema:
employee:
(Milo,30,DH,,,)
(Asmya,34,PQ,,,)
(Baljit,23,RS,,,)
(Pune,60,Astrophysics,Warriors,5466,UTA)
(Rajsathan,20,Biochemistry,Royals,1378,Stanford)
(Chennai,50,Microbiology,Superkings,7338,Hopkins)
(Mumbai,20,Applied Math,Indians,4468,UAH)
(Praj,54,RMX,,,Champaign)
(Buba,767,HD,,,Sunnyvale)
(Manku,375,MS,,,New York)
Regards
Viraj