[PIG-3981] Use Tez counter to collect per output stats

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.18.0
• Component/s: tez
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 02/Jun/14 21:03
• Updated: 26/May/17 22:35

Description

Pig currently goes to hdfs to collect those information. That will take from several hundred milliseconds to several seconds, which adds overhead for namenode and hurt performance especially for iterative queries.

Issue Links

• is blocked by: TEZ-1173 Need per output counters