Uploaded image for project: 'OODT'
  1. OODT
  2. OODT-698 Adapt OODT Process Streaming Data
  3. OODT-764

Create a Resource Manager Backend That Muliplexes

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8
    • Component/s: resource manager
    • Labels:
      None
    • Skill Level:
      Committer (Medium) - This is for regular contributors/committers

      Description

      If a user wants to run various jobs using different technologies behind the resource manager, then there needs to be a technology to choose between them.

      For example (scheduler is the key component of the resource manager backend):

      "mesos job" -> Mesos scheduler et. al.
      "spark job -> Spark scheduler et. al.
      "normal job" -> Standard resource manager scheduler et. al.
      "torque job" -> Torque scheduler et. al.

      The natural place to make the switch is based on the queue that that job is in. Thus we can create a resource manager backend that multiplexes jobs based on the queue that they are in.

      This can be accomplished by using XML configuration to map queues to backend sets, and create a custom set to multiplex between what is specified in XML. The set must include the following:

      Muxing components:
      -Scheduler
      -Batchmgr
      -Monitor

      Supporting components:
      -Backend Manager - keeps track of children (Scheduler, Monitor, Batchmgr) and which queue these sets are associated.
      -Backend repository - to setup BackendManager via config

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              starchmd@umich.edu Michael Starch
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: