Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3490

RMContainerAllocator counts failed maps towards Reduce ramp up

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed MapReduce AM to count failed maps also towards Reduce ramp up.

      Description

      The RMContainerAllocator does not differentiate between failed and successful maps while calculating whether reduce tasks are ready to launch. Failed tasks are also counted towards total completed tasks.
      Example. 4 failed maps, 10 total maps. Map%complete = 4/14 * 100 instead of being 0.

      1. MAPREDUCE-3490.patch
        13 kB
        Arun C Murthy
      2. MAPREDUCE-3490.patch
        19 kB
        Arun C Murthy
      3. MAPREDUCE-3490.patch
        19 kB
        Arun C Murthy
      4. MAPREDUCE-3490.patch
        15 kB
        Arun C Murthy
      5. MAPREDUCE-3490.patch
        12 kB
        Arun C Murthy
      6. MAPREDUCE-3490.patch
        11 kB
        Arun C Murthy
      7. MAPREDUCE-3490.patch
        7 kB
        Arun C Murthy
      8. MR-3490-alternate.patch
        3 kB
        Sharad Agarwal
      9. MR-3490-alternate1.patch
        5 kB
        Sharad Agarwal

        Activity

          People

          • Assignee:
            Sharad Agarwal
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development