Hadoop Common
  1. Hadoop Common
  2. HADOOP-1077

Race condition in fetching map outputs (might lead to hung reduces)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.1
    • Component/s: None
    • Labels:
      None

      Description

      Sometimes when a map task is lost while the map-output fetch is happening from the TT for that task, and the lost map has successfully executed on some other node, the event for that successful execution is lost at the fetching TT. The fetching TT might eventually fail to fetch the output for the lost task, but then since the event for the new run of the lost map might also have been lost, the fetching TT might hang.

      This "hung" problem was discovered while working on HADOOP-1060.

      1. 1077.2.patch
        6 kB
        Arun C Murthy
      2. 1077.patch
        7 kB
        Devaraj Das

        Issue Links

          Activity

          Devaraj Das created issue -
          Devaraj Das made changes -
          Field Original Value New Value
          Attachment 1077.patch [ 12352833 ]
          Nigel Daley made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Fix Version/s 0.12.1 [ 12312347 ]
          Arun C Murthy made changes -
          Link This issue blocks HADOOP-1060 [ HADOOP-1060 ]
          Arun C Murthy made changes -
          Attachment 1077.2.patch [ 12352874 ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Doug Cutting made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development