Pig
  1. Pig
  2. PIG-2238

Pig 0.9 error message not useful as compared to 0.8

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The below is my faulty script (note the usage of alias F) for which Pig 0.9 composes not so useful message as compared to 0.8;

      A = load 'input' using TextLoader as (doc:chararray) ;
      B = foreach A generate flatten(TOKENIZE(doc)) as myword;
      C = group B by myword parallel 30;
      D = foreach C generate group,COUNT(B) as count,SIZE(group) as size;

      E = order D by size parallel 5;
      >F = limit F 20;
      dump F;

      For this script , error message in 0.9
      >ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2240: LogicalPlanVisitor can only visit logical plan

      Error message in 0.8
      >ERROR 1000: Error during parsing. Unrecognized alias F

      1. PIG-2238-0.patch
        5 kB
        Daniel Dai
      2. PIG-2238-1.patch
        6 kB
        Daniel Dai
      3. PIG-2238-1_0.9.patch
        5 kB
        Daniel Dai

        Activity

        Vivek Padmanabhan created issue -
        Hide
        Vivek Padmanabhan added a comment -

        Looks like this issue was introduced as part of the parser changes.
        In Pig 0.9, the validation is done like below;

        In org.apache.pig.parser.AstValidator

        private void validateAliasRef(Set<String> aliases, CommonTree node, String alias)
        throws UndefinedAliasException {
        if( !aliases.contains( alias ) )

        { throw new UndefinedAliasException( input, new SourceLocation( (PigParserNode)node ), alias ); }

        }

        Here it just checks that the alias is contained inside the set "aliases", but this contains all the aliases
        and it doesnt check for the order in which they are defined in the script.

        Hence this will lead to other sort of issues like NullPointerException, if i replace the F in the above script with
        below
        F = foreach F generate *;

        Show
        Vivek Padmanabhan added a comment - Looks like this issue was introduced as part of the parser changes. In Pig 0.9, the validation is done like below; In org.apache.pig.parser.AstValidator private void validateAliasRef(Set<String> aliases, CommonTree node, String alias) throws UndefinedAliasException { if( !aliases.contains( alias ) ) { throw new UndefinedAliasException( input, new SourceLocation( (PigParserNode)node ), alias ); } } Here it just checks that the alias is contained inside the set "aliases", but this contains all the aliases and it doesnt check for the order in which they are defined in the script. Hence this will lead to other sort of issues like NullPointerException, if i replace the F in the above script with below F = foreach F generate *;
        Hide
        Daniel Dai added a comment -

        We need a NPE check at LogicalPlanBuilder:439. Unfortunately we need to change other places to allow throw exception in method signature. Attach a draft patch.

        Show
        Daniel Dai added a comment - We need a NPE check at LogicalPlanBuilder:439. Unfortunately we need to change other places to allow throw exception in method signature. Attach a draft patch.
        Daniel Dai made changes -
        Field Original Value New Value
        Attachment PIG-2238-0.patch [ 12491705 ]
        Hide
        Olga Natkovich added a comment -

        I suggest that we fix it as part of 0.9.1 since it is a regression and it makes error messages worse even though the release claimed to improve error handling

        Show
        Olga Natkovich added a comment - I suggest that we fix it as part of 0.9.1 since it is a regression and it makes error messages worse even though the release claimed to improve error handling
        Olga Natkovich made changes -
        Fix Version/s 0.9.1 [ 12317343 ]
        Daniel Dai made changes -
        Attachment PIG-2238-1.patch [ 12492203 ]
        Hide
        Thejas M Nair added a comment -

        +1

        Show
        Thejas M Nair added a comment - +1
        Hide
        Daniel Dai added a comment -

        A slightly different patch for 0.9 branch.

        Show
        Daniel Dai added a comment - A slightly different patch for 0.9 branch.
        Daniel Dai made changes -
        Attachment PIG-2238-1_0.9.patch [ 12496148 ]
        Hide
        Daniel Dai added a comment -

        Patch committed to both trunk and 0.9 branch.

        Show
        Daniel Dai added a comment - Patch committed to both trunk and 0.9 branch.
        Daniel Dai made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Assignee Daniel Dai [ daijy ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Vivek Padmanabhan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development