Pig
  1. Pig
  2. PIG-2507

Semicolon in paramenters for UDF results in parsing error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.9.1, 0.10.0, 0.11, 0.11.1
    • Fix Version/s: 0.12.0, 0.11.2
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available
    • Hadoop Flags:
      Reviewed

      Description

      If I have a semicolon in the parameter passed to a udf, the script execution will fail with a parsing error.

      a = load 'i1' as (f1:chararray);
      c = foreach a generate REGEX_EXTRACT(f1, '.;' ,1);
      dump c;

      The above script fails with the below error
      [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file test.pig, line 3, column 0> mismatched character '<EOF>' expecting '''

      Even replacing the semicolon with Unicode \u003B results in the same error.
      c = foreach a generate REGEX_EXTRACT(f1, '.\u003B',1);

      1. PIG_2507.patch
        2 kB
        Timothy Chen

        Issue Links

          Activity

          Hide
          Vivek Padmanabhan added a comment -

          Btw, the same script when run on Pig 0.8 provides the exact source location. The below is the error message fro, Pig 0.8;
          [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Lexical error at line 2, column 45. Encountered: <EOF> after : "\'.;"

          Show
          Vivek Padmanabhan added a comment - Btw, the same script when run on Pig 0.8 provides the exact source location. The below is the error message fro, Pig 0.8; [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Lexical error at line 2, column 45. Encountered: <EOF> after : "\'.;"
          Hide
          Timothy Chen added a comment -

          Fixes the semicolon issue

          Show
          Timothy Chen added a comment - Fixes the semicolon issue
          Hide
          Timothy Chen added a comment -

          I tried to find where I can add tests for loading script in via grunt, but not sure how to do that

          Show
          Timothy Chen added a comment - I tried to find where I can add tests for loading script in via grunt, but not sure how to do that
          Show
          Rohini Palaniswamy added a comment - You should be able to add tests for grunt in http://svn.apache.org/repos/asf/pig/branches/branch-0.10/test/org/apache/pig/test/TestGrunt.java
          Hide
          Alan Gates added a comment -

          Changes to the code look fine, but we definitely need a unit test to check that they work. Adding it in TestGrunt as Rohini suggested makes sense. Canceling the patch pending adding of tests.

          Show
          Alan Gates added a comment - Changes to the code look fine, but we definitely need a unit test to check that they work. Adding it in TestGrunt as Rohini suggested makes sense. Canceling the patch pending adding of tests.
          Hide
          Timothy Chen added a comment -

          Added unit test in TestGruntParser for this bug

          Show
          Timothy Chen added a comment - Added unit test in TestGruntParser for this bug
          Hide
          Timothy Chen added a comment -

          Updated the patch now to include unit test for this bug, please review again.

          Show
          Timothy Chen added a comment - Updated the patch now to include unit test for this bug, please review again.
          Hide
          Daniel Dai added a comment -

          Patch committed to trunk. Thanks Timothy!

          Show
          Daniel Dai added a comment - Patch committed to trunk. Thanks Timothy!
          Hide
          Appana Satya Harish added a comment -

          Trying Using '
          u003B', it works. I am using it on Pig 0.10.0 version.

          Show
          Appana Satya Harish added a comment - Trying Using ' u003B', it works. I am using it on Pig 0.10.0 version.
          Hide
          Cheolsoo Park added a comment -

          Committed to branch 11:
          http://svn.apache.org/viewvc?view=revision&revision=1514056

          Updated the fix version.

          Show
          Cheolsoo Park added a comment - Committed to branch 11: http://svn.apache.org/viewvc?view=revision&revision=1514056 Updated the fix version.

            People

            • Assignee:
              Timothy Chen
              Reporter:
              Vivek Padmanabhan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development