Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1198

Alternatively schedule different types of tasks in fair share scheduler

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: contrib/fair-share
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Matei has mentioned in MAPREDUCE-961 that the current scheduler will first try to launch map tasks until canLaunthTask() returns false then look for reduce tasks. This might starve reduce task. He also mention that alternatively schedule different types of tasks can solve this problem.

      1. MAPREDUCE-1198-v1.patch
        20 kB
        Scott Chen
      2. MAPREDUCE-1198-v2.patch
        29 kB
        Scott Chen
      3. MAPREDUCE-1198-v3.patch
        29 kB
        Scott Chen
      4. MAPREDUCE-1198-v4.patch
        29 kB
        Scott Chen
      5. MAPREDUCE-1198-v5.patch
        29 kB
        Scott Chen

        Activity

        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Matei Zaharia made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v5.patch [ 12425540 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v4.patch [ 12425517 ]
        Scott Chen made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Scott Chen made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v3.patch [ 12425418 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v3.patch [ 12425417 ]
        Scott Chen made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Scott Chen made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v3.patch [ 12425417 ]
        Scott Chen made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v2.patch [ 12425272 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v2.patch [ 12425271 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v2.patch [ 12425271 ]
        Scott Chen made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Scott Chen made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Scott Chen made changes -
        Attachment MAPREDUCE-1198-v1.patch [ 12425128 ]
        Scott Chen made changes -
        Priority Minor [ 4 ] Major [ 3 ]
        Scott Chen made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Minor [ 4 ]
        Scott Chen created issue -

          People

          • Assignee:
            Scott Chen
            Reporter:
            Scott Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development