1. Pig
  2. PIG-3359

Register Statements and Param Substitution in Macros


    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: parser
    • Labels:
    • Release Note:
      Register statements and parameter substitutions can be used inside macro now.


      There are some gaps in the functionality of macros that I've made a patch to address. The goal is to provide everything you'd need to make reusable algorithms libraries.

      1. You can't register udfs inside a macro
      2. Paramater substitutions aren't done inside macros
      3. Resources (including macros) should not be redundantly acquired if they are already present.

      Rohini's patch https://issues.apache.org/jira/browse/PIG-3204 should address problem 3 where Pig reparses everything every time it reads a line, but there still would be a problem if two separate files import the same macro / udf file.

      To get this working, I moved methods for registering jars/udfs and param substitution from PigServer to PigContext so they can be accessed in QueryParserDriver which processes macros (QPD was already passed a PigContext reference). Is that ok?

      1. PIG-3359-v7.diff
        68 kB
        Jonathan Packer
      2. PIG-3359-v6.diff
        63 kB
        Jonathan Packer
      3. PIG-3359-v5.diff
        58 kB
        Jonathan Packer
      4. PIG-3359-v4.diff
        57 kB
        Jonathan Packer
      5. PIG-3359-v3-test-failures.txt
        3 kB
        Cheolsoo Park
      6. PIG-3359-v3.diff
        61 kB
        Jonathan Packer
      7. PIG-3359-v2.diff
        34 kB
        Jonathan Packer
      8. PIG-3359_test.tar.gz
        253 kB
        Jonathan Packer
      9. PIG-3359-v1.diff
        35 kB
        Jonathan Packer



          • Assignee:
            Jonathan Packer
            Jonathan Packer
          • Votes:
            0 Vote for this issue
            6 Start watching this issue


            • Created: