[PIG-4057] Group All followed by CROSS with default parallelism produces wrong results

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Hadoop Flags: Reviewed
• Assignee: Daniel Dai
• Reporter: Rohini Palaniswamy
• Created: 14/Jul/14 17:52
• Updated: 21/Nov/14 05:59
• Resolved: 28/Jul/14 02:54

Description

SET default_parallel 199;
......
by_size = ...
uniq_vals = .....
grpd = group uniq_vals all;
all_vals = FOREACH grpd GENERATE uniq_vals;
cross_result = CROSS by_size, all_vals;
store cross_result into '/tmp/roh/cross/out/recipient_asns';

Job1: grpd, all_vals, cross_result (The plan does GFCross function here for
all_vals assuming cross parallelism to be 1 taking that of the current job even
though it should consider default parallelism 199 of Job 2. Parallelism of Job1
is 1 because of group all)
Job2: cross_result (Actual CROSS of by_size and all_vals)