[PIG-877] Push up filter does not account for added columns in foreach

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.4.0
• Hadoop Flags: Reviewed
• Assignee: Santhosh Muthur Srinivasan
• Reporter: Santhosh Muthur Srinivasan
• Created: 10/Jul/09 00:37
• Updated: 24/Mar/10 22:13
• Resolved: 14/Jul/09 17:42

Description

If a filter follows a foreach that produces an added column then push up filter fails with a null pointer exception.

...
x = foreach w generate $0, COUNT($1);
y = filter x by $1 > 10;

In the above example, the column in the filter's expression is an added column. As a result, the optimizer rule is not able to map it back to the input resulting in a null value. The subsequent for loop is failing due to NPE.