[MAPREDUCE-3050] YarnScheduler needs to expose Resource Usage Information

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.23.0, 2.0.0-alpha
• Fix Version/s: 0.23.0
• Component/s: mrv2, resourcemanager
• Target Version/s: 0.23.0
• Assignee: Robert Joseph Evans
• Reporter: Robert Joseph Evans
• Created: 20/Sep/11 17:05
• Updated: 10/Mar/15 04:31
• Resolved: 30/Sep/11 22:20

Description

Before the recent refactor The nodes had information in them about how much resources they were using. This information is not hidden inside SchedulerNode. Similarly resource usage information about an application, or in aggregate is only available through the Scheduler and there is not interface to pull it out.

We need to expose APIs to get Resource and Container information from the scheduler, in aggregate across the entire cluster, per application, per node, and ideally also per queue if applicable (although there are no JIRAs I am aware of that need this right now).

Issue Links

• blocks: MAPREDUCE-2738 Missing cluster level stats on the RM UI
• blocks: MAPREDUCE-2790 [MR-279] Add additional field for storing the AM/job history info on CLI
• blocks: MAPREDUCE-2789 [MR:279] Update the scheduling info on CLI