Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-944

Extend FairShare scheduler to fair-share memory usage in the cluster

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • contrib/fair-share
    • None
    • Reviewed

    Description

      The FairShare Scheduler has an extensible LoadManager API to regulate allocating new tasks on a particular TaskTracker. In similar lines, it would be nice if the FairShare Scheduler can have a pluggable policy to regulate new tasks from a particular job. This will allow one to skip scheduling tasks of a job that is eating a large percentage of memory in the cluster, i.e. fair-share of memory resources among jobs.

      Attachments

        1. LoadManager.txt
          4 kB
          Dhruba Borthakur
        2. LoadManager2.txt
          5 kB
          Dhruba Borthakur

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-961 ResourceAwareLoadManager to dynamically decide new tasks based on current CPU/memory load on TaskTracker(s)

          • Major - Major loss of function.
          • Open

          Activity

            People

              dhruba Dhruba Borthakur
              dhruba Dhruba Borthakur
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: