Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5844

Add a configurable delay to reducer-preemption

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.5.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We observed cases where the reducer preemption makes the job finish much later, and the preemption does not seem to be necessary since after preemption both the preempted reducer and the mapper are assigned immediately--meaning that there was already enough space for the mapper.

      The logic for triggering preemption is at RMContainerAllocator::preemptReducesIfNeeded
      The preemption is triggered if the following is true:

      headroom +  am * |m| + pr * |r| < mapResourceRequest
      

      where am: number of assigned mappers, |m| is mapper size, pr is number of reducers being preempted, and |r| is the reducer size.

      The original idea apparently was that if headroom is not big enough for the new mapper requests, reducers should be preempted. This would work if the job is alone in the cluster. Once we have queues, the headroom calculation becomes more complicated and it would require a separate headroom calculation per queue/job.

      So, as a result headroom variable is kind of given up currently: headroom is always set to 0 What this implies to the speculation is that speculation becomes very aggressive, not considering whether there is enough space for the mappers or not.

        Attachments

        1. MAPREDUCE-5844.patch
          28 kB
          Maysam Yabandeh
        2. MAPREDUCE-5844.patch
          27 kB
          Maysam Yabandeh
        3. MAPREDUCE-5844.patch
          28 kB
          Maysam Yabandeh
        4. MAPREDUCE-5844.patch
          28 kB
          Maysam Yabandeh
        5. MAPREDUCE-5844.patch
          28 kB
          Maysam Yabandeh
        6. MAPREDUCE-5844.patch
          27 kB
          Maysam Yabandeh
        7. MAPREDUCE-5844.patch
          20 kB
          Maysam Yabandeh
        8. MAPREDUCE-5844.patch
          18 kB
          Maysam Yabandeh
        9. MAPREDUCE-5844.patch
          7 kB
          Maysam Yabandeh
        10. MAPREDUCE-5844-branch-2.patch
          28 kB
          Maysam Yabandeh

          Issue Links

            Activity

              People

              • Assignee:
                maysamyabandeh Maysam Yabandeh
                Reporter:
                maysamyabandeh Maysam Yabandeh
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: