Uploaded image for project: 'Myriad'
  1. Myriad
  2. MYRIAD-103

Optimization: Reuse Mesos offer across YARN containers

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Executor, Scheduler
    • None

    Description

      In the fine grained scheduling implementation, when YARN scheduler allocates a container, Myriad launches a pseudo Mesos task for it by specifying its resource allocation. When the container completes, a task finished update is sent to Mesos so that the pseudo task is stopped and its resources reclaimed. When there are pending tasks in the YARN scheduler’s pipeline, it would be beneficial to hold on to these resources and see if it can be used again. This avoids the overhead of giving away resources and waiting for Mesos to offer it back.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rkannan82 Kannan Rajah
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated: