[PIG-1525] Incorrect data generated by diff of SUM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.8.0
• Hadoop Flags: Reviewed
• Assignee: Richard Ding
• Reporter: Richard Ding
• Created: 29/Jul/10 21:59
• Updated: 17/Dec/10 22:45
• Resolved: 09/Aug/10 17:48

Description

Given data;

input1:

id9     0

input2:

id8     1
id9     1

Pig script

A = LOAD 'input1' AS (id:chararray, val:long);
B = LOAD 'input2' AS (id:chararray, val:long);
C = COGROUP A BY id, B BY id;
D = FOREACH C GENERATE group, SUM(B.val), SUM(A.val), (SUM(A.val) - SUM(B.val));
dump D;

generates incorrect data:

(id8,1L,,)
(id9,1L,0L,-2L)

The workaround is to replace the FOREACH statement with

D = FOREACH C GENERATE group, SUM(B.val) as b, SUM(A.val) as a;
E = FOREACH D GENERATE $0, b, a, (a-b);