[HADOOP-3930] Decide how to integrate scheduler info into CLI and job tracker web page

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.19.0
• Fix Version/s: 0.19.0
• Hadoop Flags: Reviewed
• Release Note: Changed TaskScheduler to expose API for Web UI and Command Line Tool.
• Assignee: Sreekanth Ramakrishnan
• Reporter: Matei Zaharia
• Created: 09/Aug/08 00:08
• Updated: 08/Jul/09 16:52
• Resolved: 18/Sep/08 19:04

Description

We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to provide info to display on the JobTracker web interface and in the CLI. The main things needed seem to be:

• A way for schedulers to provide info to show in a column on the web UI and in the CLI - something as simple as a single string, or a map<string, int> for multiple parameters.
• Some sorting order for jobs - maybe a method to sort a list of jobs.

Let's figure out what the best way to do this is and implement it in the existing schedulers.

My first-order proposal at an API: Augment the TaskScheduler with

• public Map<String, String> getSchedulingInfo(JobInProgress job) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of jobs.
• public Map<String, String> getSchedulingInfo(String queue) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of queues.
• public Collection<JobInProgress> getJobs(String queueName) – returns the list of jobs in a given queue, sorted by a scheduler-specific order (the order it wants to run them in / schedule the next task in / etc).
• public List<String> getQueues();

Issue Links

• blocks: HADOOP-3445 Implementing core scheduler functionality in Resource Manager (V1) for Hadoop
• blocks: HADOOP-3746 A fair sharing job scheduler
• duplicates: HADOOP-3699 Create a UI for viewing configured queues and related information
• is part of: HADOOP-3444 Implementing a Resource Manager (V1) for Hadoop
• is related to: HADOOP-4213 NPE in TestLimitTasksPerJobTaskScheduler