Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2507

Semicolon in parameters for UDF results in parsing error

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.9.1, 0.10.0, 0.11, 0.11.1
    • Fix Version/s: 0.12.0, 0.11.2
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available
    • Hadoop Flags:
      Reviewed

      Description

      If I have a semicolon in the parameter passed to a udf, the script execution will fail with a parsing error.

      a = load 'i1' as (f1:chararray);
      c = foreach a generate REGEX_EXTRACT(f1, '.;' ,1);
      dump c;

      The above script fails with the below error
      [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file test.pig, line 3, column 0> mismatched character '<EOF>' expecting '''

      Even replacing the semicolon with Unicode \u003B results in the same error.
      c = foreach a generate REGEX_EXTRACT(f1, '.\u003B',1);

      1. PIG_2507.patch
        2 kB
        Timothy Chen

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user deepika087 closed the pull request at:

          https://github.com/apache/pig/pull/32

          Show
          githubbot ASF GitHub Bot added a comment - Github user deepika087 closed the pull request at: https://github.com/apache/pig/pull/32
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user deepika087 opened a pull request:

          https://github.com/apache/pig/pull/32

          Fixed the PIG-2507: related to grunt error

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/deepika087/pig branch-0.7

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/pig/pull/32.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #32


          commit 050a4697408acbe62a0d87398d35f821c36479ad
          Author: Deepika Anand <deepika@deepikas-macbook-pro-2.local>
          Date: 2017-10-02T07:23:50Z

          Fixed the PIG-2507: related to grunt error


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user deepika087 opened a pull request: https://github.com/apache/pig/pull/32 Fixed the PIG-2507 : related to grunt error You can merge this pull request into a Git repository by running: $ git pull https://github.com/deepika087/pig branch-0.7 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/pig/pull/32.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #32 commit 050a4697408acbe62a0d87398d35f821c36479ad Author: Deepika Anand <deepika@deepikas-macbook-pro-2.local> Date: 2017-10-02T07:23:50Z Fixed the PIG-2507 : related to grunt error
          Hide
          cheolsoo Cheolsoo Park added a comment -

          Committed to branch 11:
          http://svn.apache.org/viewvc?view=revision&revision=1514056

          Updated the fix version.

          Show
          cheolsoo Cheolsoo Park added a comment - Committed to branch 11: http://svn.apache.org/viewvc?view=revision&revision=1514056 Updated the fix version.
          Hide
          satyaharish Appana Satya Harish added a comment -

          Trying Using '
          u003B', it works. I am using it on Pig 0.10.0 version.

          Show
          satyaharish Appana Satya Harish added a comment - Trying Using ' u003B', it works. I am using it on Pig 0.10.0 version.
          Hide
          daijy Daniel Dai added a comment -

          Patch committed to trunk. Thanks Timothy!

          Show
          daijy Daniel Dai added a comment - Patch committed to trunk. Thanks Timothy!
          Hide
          tnachen Timothy Chen added a comment -

          Updated the patch now to include unit test for this bug, please review again.

          Show
          tnachen Timothy Chen added a comment - Updated the patch now to include unit test for this bug, please review again.
          Hide
          tnachen Timothy Chen added a comment -

          Added unit test in TestGruntParser for this bug

          Show
          tnachen Timothy Chen added a comment - Added unit test in TestGruntParser for this bug
          Hide
          alangates Alan Gates added a comment -

          Changes to the code look fine, but we definitely need a unit test to check that they work. Adding it in TestGrunt as Rohini suggested makes sense. Canceling the patch pending adding of tests.

          Show
          alangates Alan Gates added a comment - Changes to the code look fine, but we definitely need a unit test to check that they work. Adding it in TestGrunt as Rohini suggested makes sense. Canceling the patch pending adding of tests.
          Show
          rohini Rohini Palaniswamy added a comment - You should be able to add tests for grunt in http://svn.apache.org/repos/asf/pig/branches/branch-0.10/test/org/apache/pig/test/TestGrunt.java
          Hide
          tnachen Timothy Chen added a comment -

          I tried to find where I can add tests for loading script in via grunt, but not sure how to do that

          Show
          tnachen Timothy Chen added a comment - I tried to find where I can add tests for loading script in via grunt, but not sure how to do that
          Hide
          tnachen Timothy Chen added a comment -

          Fixes the semicolon issue

          Show
          tnachen Timothy Chen added a comment - Fixes the semicolon issue
          Hide
          vivekp Vivek Padmanabhan added a comment -

          Btw, the same script when run on Pig 0.8 provides the exact source location. The below is the error message fro, Pig 0.8;
          [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Lexical error at line 2, column 45. Encountered: <EOF> after : "\'.;"

          Show
          vivekp Vivek Padmanabhan added a comment - Btw, the same script when run on Pig 0.8 provides the exact source location. The below is the error message fro, Pig 0.8; [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Lexical error at line 2, column 45. Encountered: <EOF> after : "\'.;"

            People

            • Assignee:
              tnachen Timothy Chen
              Reporter:
              vivekp Vivek Padmanabhan
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development