[OODT-764] Create a Resource Manager Backend That Muliplexes

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.8
• Component/s: resource manager
• Skill Level: Committer (Medium) - This is for regular contributors/committers
• Assignee: Unassigned
• Reporter: Michael Starch
• Created: 24/Oct/14 19:39
• Updated: 05/Nov/14 21:46
• Resolved: 05/Nov/14 21:45

Description

If a user wants to run various jobs using different technologies behind the resource manager, then there needs to be a technology to choose between them.

For example (scheduler is the key component of the resource manager backend):

"mesos job" -> Mesos scheduler et. al.
"spark job -> Spark scheduler et. al.
"normal job" -> Standard resource manager scheduler et. al.
"torque job" -> Torque scheduler et. al.

The natural place to make the switch is based on the queue that that job is in. Thus we can create a resource manager backend that multiplexes jobs based on the queue that they are in.

This can be accomplished by using XML configuration to map queues to backend sets, and create a custom set to multiplex between what is specified in XML. The set must include the following:

Muxing components:
-Scheduler
-Batchmgr
-Monitor

Supporting components:
-Backend Manager - keeps track of children (Scheduler, Monitor, Batchmgr) and which queue these sets are associated.
-Backend repository - to setup BackendManager via config