[PIG-5261] Support for macro based property definitions in Pig

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.16.0
• Component/s: parser
• Patch Info: Patch Available
• Flags: Patch
• Assignee: Unassigned
• Reporter: Sridhar Sailappan
• Created: 16/Jun/17 00:06
• Updated: 16/Jun/17 00:06

Description

The attached patch provides support for property configurations with macro based definitions (relying on variable expansion of Hadoop Configuration).
 
e.g., Consider the following definition of properties for HDFS.
1) The property "fs.defaultFS" is defined to have value "${fs.defaultFS.dev}"
2) The custom property "fs.defaultFS.dev" is defined to have value "hdfs://dev-sandbox.domain.com:8020"
 
Now, when referencing the property "fs.defaultFS", the value should be resolved to "hdfs://dev-sandbox.domain.com:8020".
 
This is typical variable expansion that is expected in Hadoop Configuration. Refer: https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/conf/Configuration.html
This functionality of variable expansion is not available with Pig. The attached patch (on branch-0.16) addresses this need.