Details

Type: Task

Status: Closed

Priority: Major

Resolution: Fixed

Affects Version/s: None

Fix Version/s: 0.9.0

Component/s: None

Labels:None

Release Note:
Description
We should consider using logic for merging inner schema in case of the two different types of union.
In case of 'default union', it merges the two inner schema of bags/tuples by position if the number of fields are same and the corresponding types are compatible.
In case of 'union onschema', it considers tuple/bag with different innerschema to be incompatible types.
Activity
 All
 Comments
 Work Log
 History
 Activity
 Transitions
Thejas M Nair
created issue 
Alan Gates
made changes 
Field  Original Value  New Value 

Assignee  Alan Gates [ alangates ] 
Alan Gates
made changes 
Assignee  Alan Gates [ alangates ]  Daniel Dai [ daijy ] 
Daniel Dai
made changes 
Attachment  PIG15361.patch [ 12470199 ] 
Daniel Dai
made changes 
Release Note 
The behavior of union and uniononschema converges after this patch:
1. Union on relation of two different size result a null schema (union only): A: (a1:long, a2:long) B: (b1:long, b2:long, b3:long) A union B: null 2. Union column of incompatible type result a bytearray type: A: (a1:long, a2:long) B: (b1:(b11:long, b12:long), b2:long) A union B: (a1:bytearray, a2:long) 3. Union column of compatible type will produce a escalate the type. The priority is chararray > double > float > long > int: A: (a1:int, a2:double) B: (b1:float, b2:chararray) A union B: (a1:float, a2:chararray) 4. Union different inner type result an empty complex type: A: (a1:(a11:long, a12:int), a2:{(a21:charray, a22:int)}) B: (b1:(b11:int, b12:int), b2:{(b21:int, b22:int)}) A union B: (a1:(), a2:{()}) 5. Always take the alias of first relation as the alias of unioned relation field 
Daniel Dai
made changes 
Release Note 
The behavior of union and uniononschema converges after this patch:
1. Union on relation of two different size result a null schema (union only): A: (a1:long, a2:long) B: (b1:long, b2:long, b3:long) A union B: null 2. Union column of incompatible type result a bytearray type: A: (a1:long, a2:long) B: (b1:(b11:long, b12:long), b2:long) A union B: (a1:bytearray, a2:long) 3. Union column of compatible type will produce a escalate the type. The priority is chararray > double > float > long > int: A: (a1:int, a2:double) B: (b1:float, b2:chararray) A union B: (a1:float, a2:chararray) 4. Union different inner type result an empty complex type: A: (a1:(a11:long, a12:int), a2:{(a21:charray, a22:int)}) B: (b1:(b11:int, b12:int), b2:{(b21:int, b22:int)}) A union B: (a1:(), a2:{()}) 5. Always take the alias of first relation as the alias of unioned relation field 
The behavior of union and uniononschema converges after this patch:
1. Union on relation of two different size result a null schema (union only): A: (a1:long, a2:long) B: (b1:long, b2:long, b3:long) A union B: null 2. Union column of incompatible type result a bytearray type: A: (a1:long, a2:long) B: (b1:(b11:long, b12:long), b2:long) A union B: (a1:bytearray, a2:long) 3. Union column of compatible type will produce a escalate the type. The priority is chararray > double > float > long > int > bytearray: A: (a1:int, a2:double, a3:int) B: (b1:float, b2:chararray, b3:bytearray) A union B: (a1:float, a2:chararray, a3:int) 4. Union different inner type result an empty complex type: A: (a1:(a11:long, a12:int), a2:{(a21:charray, a22:int)}) B: (b1:(b11:int, b12:int), b2:{(b21:int, b22:int)}) A union B: (a1:(), a2:{()}) 5. Always take the alias of first relation as the alias of unioned relation field 
Daniel Dai
made changes 
Attachment  PIG15362.patch [ 12470493 ] 
Daniel Dai
made changes 
Attachment  PIG15363.patch [ 12470620 ] 
Daniel Dai
made changes 
Attachment  PIG15364.patch [ 12471053 ] 
Daniel Dai
made changes 
Status  Open [ 1 ]  Resolved [ 5 ] 
Resolution  Fixed [ 1 ] 
Olga Natkovich
made changes 
Status  Resolved [ 5 ]  Closed [ 6 ] 