Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
0.8.0, 0.9.0
-
None
-
None
Description
For the following pig script,
grunt> A = load 'file' as ( a : tuple( x, y, z ), b, c );
grunt> B = foreach A generate flatten( $0 ), b, c;
grunt> describe B;
B:
Alias B has a clear schema.
However, on the backend, for a row if $0 happens to be null, then output tuple become something like
(null, b_value, c_value), which is obviously inconsistent with the schema. The behaviour is confirmed by pig code inspection.
This inconsistency corrupts data because of position shifts. Expected output row should be something like
(null, null, null, b_value, c_value).
Attachments
Attachments
Issue Links
- relates to
-
PIG-5201 Null handling on FLATTEN
- Resolved