-
Type:
Sub-task
-
Status: Closed
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.8
-
Component/s: resource manager
-
Labels:None
-
Skill Level:Committer (Medium) - This is for regular contributors/committers
If a user wants to run various jobs using different technologies behind the resource manager, then there needs to be a technology to choose between them.
For example (scheduler is the key component of the resource manager backend):
"mesos job" -> Mesos scheduler et. al.
"spark job -> Spark scheduler et. al.
"normal job" -> Standard resource manager scheduler et. al.
"torque job" -> Torque scheduler et. al.
The natural place to make the switch is based on the queue that that job is in. Thus we can create a resource manager backend that multiplexes jobs based on the queue that they are in.
This can be accomplished by using XML configuration to map queues to backend sets, and create a custom set to multiplex between what is specified in XML. The set must include the following:
Muxing components:
-Scheduler
-Batchmgr
-Monitor
Supporting components:
-Backend Manager - keeps track of children (Scheduler, Monitor, Batchmgr) and which queue these sets are associated.
-Backend repository - to setup BackendManager via config