Uploaded image for project: 'OODT'
  1. OODT
  2. OODT-698 Adapt OODT Process Streaming Data
  3. OODT-764

Create a Resource Manager Backend That Muliplexes

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.8
    • resource manager
    • None
    • Committer (Medium) - This is for regular contributors/committers

    Description

      If a user wants to run various jobs using different technologies behind the resource manager, then there needs to be a technology to choose between them.

      For example (scheduler is the key component of the resource manager backend):

      "mesos job" -> Mesos scheduler et. al.
      "spark job -> Spark scheduler et. al.
      "normal job" -> Standard resource manager scheduler et. al.
      "torque job" -> Torque scheduler et. al.

      The natural place to make the switch is based on the queue that that job is in. Thus we can create a resource manager backend that multiplexes jobs based on the queue that they are in.

      This can be accomplished by using XML configuration to map queues to backend sets, and create a custom set to multiplex between what is specified in XML. The set must include the following:

      Muxing components:
      -Scheduler
      -Batchmgr
      -Monitor

      Supporting components:
      -Backend Manager - keeps track of children (Scheduler, Monitor, Batchmgr) and which queue these sets are associated.
      -Backend repository - to setup BackendManager via config

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            starchmd@umich.edu Michael Starch
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment