[TEZ-2951] Progressive allocation of buffers for Unordered Partitioned Output

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Target Version/s: 0.8.6
• Assignee: Unassigned
• Reporter: Siddharth Seth
• Created: 19/Nov/15 00:07
• Updated: 14/Mar/17 03:40

Description

Similar to TEZ-2244. In the case of UnorderedPartitionOutput - the default is to use 2 buffers. This can be changed to a higher value when using pipelined shuffle - and have memory allocated only when required.