[PIG-1172] PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in Join

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.6.0
• Component/s: impl
• Hadoop Flags: Reviewed
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 24/Dec/09 22:13
• Updated: 14/Apr/10 00:38
• Resolved: 05/Jan/10 03:53

Description

Currently the following script will push B below D. But we will use fattened column in the join, we cannot push that.

A = load '1.txt' as (bg:bag

{t:tuple(a0,a1)}

);
B = FOREACH A generate flatten($0);
C = load '3.txt' AS (c0, c1);
D = JOIN B by a1, C by c1;
E = limit D 10;
explain E;

Issue Links

• is related to: PIG-1374 PushDownForeachFlatten shall not push ForEach below Join if the flattened fields is used in the next statement