Hadoop Common
  1. Hadoop Common
  2. HADOOP-4788

Set mapred.fairscheduler.assignmultiple to true by default

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      Pretty much all users of the fair scheduler are using the "assignmultiple" feature which lets it launch reduce tasks at the same time as map tasks (on the same heartbeats), leading to reduces starting faster. This is also going to happen in Hadoop by default in HADOOP-3136. So we should set it to true by default and perhaps even remove the config parameter.

        Activity

        Matei Zaharia created issue -
        Matei Zaharia made changes -
        Field Original Value New Value
        Attachment hadoop-4788-v0.patch [ 12402612 ]
        Matei Zaharia made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Matei Zaharia [ matei ]
        Matei Zaharia made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note Fair scheduler assigns both a map and a reduce on each heartbeat by default (HADOOP-4788).
        Fix Version/s 0.21.0 [ 12313563 ]
        Resolution Fixed [ 1 ]
        Owen O'Malley made changes -
        Component/s contrib/fair-share [ 12312456 ]
        Robert Chansler made changes -
        Release Note Fair scheduler assigns both a map and a reduce on each heartbeat by default (HADOOP-4788).
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Matei Zaharia
            Reporter:
            Matei Zaharia
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development