Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-108

Blacklisted hosts may not be able to serve map outputs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After a node fails 4 mappers (tasks), it is added to blacklist thus it will no longer accept tasks.
      But, it will continue serve the map outputs of any mappers that ran successfully there.
      However, the node may not be able serve the map outputs either.
      This will cause the reducers to mark the corresponding map outputs as from slow hosts,
      but continue to try to get the map outputs from that node.
      This may lead to waiting forever.

      1. HADOOP-2175-v1.patch
        2 kB
        Amar Kamat
      2. HADOOP-2175-v1.1.patch
        14 kB
        Amar Kamat
      3. HADOOP-2175-v2.patch
        11 kB
        Amar Kamat
      4. HADOOP-2175-v2.patch
        11 kB
        Amar Kamat

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Amar Kamat
              Reporter:
              Runping Qi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development