[PIG-2086] grunt parser fails for: load .. as \n (b:bag{});

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: grunt

Environment

mac 10.5.8

• Assignee: Unassigned
• Reporter: Woody Anderson
• Created: 21/May/11 03:35
• Updated: 12/Mar/12 22:57

Description

this snippet fails:

IN4 = load '$in' using
    com.zzz.Storage() as
    ( inpt:bag{} );

this works (as on same line as semi-colon)

IN4 = load '$in' using
    com.zzz.Storage()
    as ( inpt:bag{} );

this is the grunt error:
2011-05-20 20:19:34,934 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file loadstore.pig, line 68, column 16> mismatched input ';' expecting RIGHT_PAREN

this only happens in cases where the types of the fields are complex e.g. bags/tuples
eg. change the type of inpt to be chararray and it will parse.

this is very strange! and i spent hours debugging my schema writing skills and reading QueryParser.g before simply trying "as (expr);" on the same line.

all of my scripts had been written with the lines split the other way (with lots of ctor args and as-clause elements: hence the line breaks), this is not an issue if i don't load complicated types, but it fails in this particular case.
This is quite unexpected and seems to be undocumented and a bug imho.
i don't know enough about antlr (i was a javacc person) to make sense of why this would be an issue for the parser b/c the grammar looks good assuming newline is basically whitespace.

though i can't figure out how newlines are treated in the grammar, there does not seem to be a newline routine ala https://supportweb.cs.bham.ac.uk/documentation/tutorials/docsystem/build/tutorials/antlr/antlr.html

I'm going to assume the grammar author is much more sophisticated than that tutorial and knows how to fix this.

Issue Links

• relates to: PIG-2414 Syntax error when line break and tab involved