[PIG-2931] $ signs in the replacement string make parameter substitution fail

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.11

Release Note

This changes the behavior of parameter substitution so that dollar signs in the replacement string are no longer treated as references to captured subsequences, but they are now treated as a literal replacement string. Therefore, this will cause backward compatibility issues for users who rely on the previous behavior of parameter substitution.

• Assignee: Cheolsoo Park
• Reporter: Cheolsoo Park
• Created: 25/Sep/12 05:40
• Updated: 22/Feb/13 04:54
• Resolved: 19/Oct/12 00:06

Description

To reproduce the issue, use the following pig script:

test.pig

a = load 'data';
b = filter by $FILTER;

and run the following command:

pig -x local -dryrun -f test.pig -p FILTER="(\$0 == 'a')"

This generates the following script:

test.pig.substituted

a = load 'data';
b = filter by ($FILTER == 'a');

However this should be:

a = load 'data';
b = filter by ($0 == 'a');

This is because Pig calls replaceFirst() with a replacement string that include a $ sign as follows:

"$FILTER".replaceFirst("\\$FILTER", "($0 == 'a')"));

To treat $ signs as literals in the replacement string, we must escape them. Please see the Java doc for Matcher class for explanation:

Note that backslashes () and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.

Issue Links

• is duplicated by: PIG-2074 When passing a parameter to Pig, if the value contains $ it has to be escaped because of a bug in PrecprocessorContext