Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3490

RMContainerAllocator counts failed maps towards Reduce ramp up

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed MapReduce AM to count failed maps also towards Reduce ramp up.
    • Target Version/s:

      Description

      The RMContainerAllocator does not differentiate between failed and successful maps while calculating whether reduce tasks are ready to launch. Failed tasks are also counted towards total completed tasks.
      Example. 4 failed maps, 10 total maps. Map%complete = 4/14 * 100 instead of being 0.

      1. MAPREDUCE-3490.patch
        13 kB
        Arun C Murthy
      2. MAPREDUCE-3490.patch
        19 kB
        Arun C Murthy
      3. MAPREDUCE-3490.patch
        19 kB
        Arun C Murthy
      4. MAPREDUCE-3490.patch
        15 kB
        Arun C Murthy
      5. MAPREDUCE-3490.patch
        12 kB
        Arun C Murthy
      6. MAPREDUCE-3490.patch
        11 kB
        Arun C Murthy
      7. MAPREDUCE-3490.patch
        7 kB
        Arun C Murthy
      8. MR-3490-alternate.patch
        3 kB
        Sharad Agarwal
      9. MR-3490-alternate1.patch
        5 kB
        Sharad Agarwal

        Activity

          People

          • Assignee:
            Sharad Agarwal
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development