Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1129

Assign multiple Map and Reduce tasks in Fairscheduler

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.20.1
    • Fix Version/s: None
    • Component/s: contrib/fair-share
    • Labels:
      None

      Description

      In Hadoop-0.20, the period of heartbeat becomes much longer.
      Fairscheduler assigns at most one Map and one Reduce task per heartbeat.
      This makes the cluster become very inefficient. Often time only half of the slots are used.

      One idea is that we make Fairscheduler detect this situation (cluster under used) and scheduler more tasks in a heartbeat.
      Thoughts?

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Scott Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development