[PIG-3359] Register Statements and Param Substitution in Macros

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.12.0
• Component/s: parser
• Release Note: Register statements and parameter substitutions can be used inside macro now.
• Assignee: Jonathan Packer
• Reporter: Jonathan Packer
• Created: 20/Jun/13 20:59
• Updated: 18/May/16 19:26
• Resolved: 06/Aug/13 05:45

Description

There are some gaps in the functionality of macros that I've made a patch to address. The goal is to provide everything you'd need to make reusable algorithms libraries.

1. You can't register udfs inside a macro
2. Paramater substitutions aren't done inside macros
3. Resources (including macros) should not be redundantly acquired if they are already present.

Rohini's patch https://issues.apache.org/jira/browse/PIG-3204 should address problem 3 where Pig reparses everything every time it reads a line, but there still would be a problem if two separate files import the same macro / udf file.

To get this working, I moved methods for registering jars/udfs and param substitution from PigServer to PigContext so they can be accessed in QueryParserDriver which processes macros (QPD was already passed a PigContext reference). Is that ok?

Issue Links

• breaks: PIG-4897 Scope of param substitution for run/exec commands
• breaks: PIG-4896 Param substitution ignored when redefined
• relates to: PIG-4880 Overlapping of parameter substitution names inside&outside a macro fails with NPE