[PIG-4842] Collected group doesn't work in some cases

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: Xianda Ke
• Reporter: Xianda Ke
• Created: 18/Mar/16 07:08
• Updated: 21/Jun/17 09:18
• Resolved: 31/Mar/16 02:31

Description

Scenario:
1. input data:
cat collectedgroup1

1
1
2

2. pig script:

A = LOAD 'collectedgroup1' USING myudfs.DummyCollectableLoader() AS (id);
B = GROUP A by $0 USING 'collected';
C = GROUP B by $0 USING 'collected';
DUMP C;

The expected output:

(1,{(1,{(1),(1)})})
(2,{(2,{(2)})})

The actual output:

(1,{(1,{(1),(1)})})
(1,)
(2,{(2,{(2)})})

Issue Links

• Is contained by: PIG-4876 OutputConsumeIterator can't handle the last buffered tuples for some Operators