[TEZ-3831] Reduce Unordered memory needed for storing empty completed events

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.1
• Assignee: Jonathan Turner Eagles
• Reporter: Jonathan Turner Eagles
• Created: 13/Sep/17 21:48
• Updated: 05/Dec/18 00:44
• Resolved: 13/Sep/17 22:28

Description

the completedInputs blocking queue is used to store inputs for the UnorderedKVReader to consume. With Auto-reduce parallelism enabled and nearly all empty inputs, the reader can't prune the empty events from the blocking queue fast enough to keep up. In my scenario, an OOM occurred.