Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-108

Blacklisted hosts may not be able to serve map outputs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After a node fails 4 mappers (tasks), it is added to blacklist thus it will no longer accept tasks.
      But, it will continue serve the map outputs of any mappers that ran successfully there.
      However, the node may not be able serve the map outputs either.
      This will cause the reducers to mark the corresponding map outputs as from slow hosts,
      but continue to try to get the map outputs from that node.
      This may lead to waiting forever.

        Attachments

        1. HADOOP-2175-v1.patch
          2 kB
          Amar Kamat
        2. HADOOP-2175-v1.1.patch
          14 kB
          Amar Kamat
        3. HADOOP-2175-v2.patch
          11 kB
          Amar Kamat
        4. HADOOP-2175-v2.patch
          11 kB
          Amar Kamat

          Issue Links

            Activity

              People

              • Assignee:
                amar_kamat Amar Kamat
                Reporter:
                runping Runping Qi
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: