[TEZ-1094] Support pipelined data transfer for Unordered Output

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Target Version/s: 0.7.0
• Hadoop Flags: Reviewed
• Assignee: Rajesh Balamohan
• Reporter: Siddharth Seth
• Created: 01/May/14 21:16
• Updated: 12/Mar/15 20:34
• Resolved: 12/Mar/15 09:12

Description

For unsorted output (and possibly for sorted output), it should be possible to send data in small batches instead of waiting for everything to be generated before transmitting. For now, planning on getting started with UnsortedOutput / Input pairs.

Issue Links

• relates to: TEZ-1991 Introduce additional DataMovementEvent types
• relates to: TEZ-1992 Move event tracking into Edges instead of Tasks
• relates to: TEZ-2001 Support pipelined data transfer for ordered output