[PIG-4483] Pig on Tez output statistics shows storing to same directory twice for union

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 0.15.0
• Hadoop Flags: Reviewed
• Assignee: Rohini Palaniswamy
• Reporter: Rohini Palaniswamy
• Created: 25/Mar/15 12:24
• Updated: 07/Jun/15 03:48
• Resolved: 02/Apr/15 16:53

Description

For the below script

A = LOAD 'data1';
B = LOAD 'data2';
C = UNION A, B;
STORE C into 'data3';

Output message is shown as below due to vertex group and storing from separate vertices.

Successfully stored 10 records (xxx bytes) in: "data3"
Successfully stored 20 records (yyy bytes) in: "data3"

Even though it is correct it can be confusing for users and they have to sum it up before comparing to Pig on MR output message. OutputStats with same filename should be combined and shown as

Successfully stored 30 records (xxx bytes) in: "data3"

Issue Links

• incorporates: PIG-4487 Pig on Tez gives wrong success message on failure in case of multiple outputs