Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3490

RMContainerAllocator counts failed maps towards Reduce ramp up

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed MapReduce AM to count failed maps also towards Reduce ramp up.

      Description

      The RMContainerAllocator does not differentiate between failed and successful maps while calculating whether reduce tasks are ready to launch. Failed tasks are also counted towards total completed tasks.
      Example. 4 failed maps, 10 total maps. Map%complete = 4/14 * 100 instead of being 0.

        Attachments

        1. MAPREDUCE-3490.patch
          13 kB
          Arun C Murthy
        2. MAPREDUCE-3490.patch
          19 kB
          Arun C Murthy
        3. MAPREDUCE-3490.patch
          19 kB
          Arun C Murthy
        4. MR-3490-alternate1.patch
          5 kB
          Sharad Agarwal
        5. MR-3490-alternate.patch
          3 kB
          Sharad Agarwal
        6. MAPREDUCE-3490.patch
          15 kB
          Arun C Murthy
        7. MAPREDUCE-3490.patch
          12 kB
          Arun C Murthy
        8. MAPREDUCE-3490.patch
          11 kB
          Arun C Murthy
        9. MAPREDUCE-3490.patch
          7 kB
          Arun C Murthy

          Activity

            People

            • Assignee:
              sharadag Sharad Agarwal
              Reporter:
              sseth Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: