[TEZ-719] Add a memory manager for Containers

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Siddharth Seth
• Reporter: Siddharth Seth
• Created: 09/Jan/14 23:36
• Updated: 09/Jan/14 23:43

Description

Tasks running in containers typically have multiple Inputs / Outputs.

At least the current set of Inputs / Outputs assume they have the entire JVM memory available to them.

A container level memory controller could serve requests from each of these Inputs / Outputs - and potentially the Processor itself at some point.

The current set of Inputs / Outputs also need to change to work with this / ensure they have limited JVM resources.
This could be configured by the DAG submitter itself - but the framework (for it's own library pieces at least) could determine this.

Issue Links

• is required by: PIG-3659 Memory management for each vertex