[PIG-579] Adding newlines to format foreach statement with constants causes parse errors

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.2.0
• Component/s: impl
• Assignee: Xuefu Zhang
• Reporter: David Ciemiewicz
• Created: 24/Dec/08 19:30
• Updated: 05/Oct/11 00:01

Description

The following code example files with parse errors on step D:

A = LOAD 'student_data' AS (name: chararray, age: int, gpa: float);

B = LOAD 'voter_data' AS (name: chararray, age: int, registration: chararray, contributions: float);

C = COGROUP A BY name, B BY name;

D = FOREACH C GENERATE
        group,
        flatten((not IsEmpty(A) ? A : (bag{tuple(chararray, int, float)}){(null, null, null)})),
        flatten((not IsEmpty(B) ? B : (bag{tuple(chararray, int, chararray, float)}){(null,null,null, null)}));

dump D;

I get the parse error:
Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Encountered "not IsEmpty ( A ) ? A : ( bag

{ tuple ( chararray , int , float ) }

;" at line 9, column 18.
Was expecting one of:
"(" ...
"-" ...
"tuple" ...
"bag" ...
"map" ...
"int" ...
"long" ...
...
However, if I simply remove the new lines from statement D and make it:

D = FOREACH C GENERATE group, flatten((not IsEmpty(A) ? A : (bag{tuple(chararray, int, float)}){(null, null, null)})), flatten((not IsEmpty(B) ? B : (bag{tuple(chararray, int, chararray, float)}){(null,null,null, null)}));

Issue Links

• depends upon: PIG-1618 Switch to new parser generator technology