Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4607

Race condition in ReduceTask completion can result in Task being incorrectly failed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Problem reported by chackaravarthy in MAPREDUCE-4252

      This problem has been handled when speculative task launched for map task and other attempt got failed (not killed)
      Can the similar kind of scenario can happen in case of reduce task?
      Consider the following scenario for reduce task in case of speculation (one attempt got killed):
      1. A task attempt is started.
      2. A speculative task attempt for the same task is started.
      3. The first task attempt completes and causes the task to transition to SUCCEEDED.
      4. Then speculative task attempt will be killed because of the completion of first attempt.
      As a result, internal error will be thrown from this attempt (TaskImpl.MapRetroactiveKilledTransition) and hence task attempt failure leads to job failure.
      TaskImpl.MapRetroactiveKilledTransition
      if (!TaskType.MAP.equals(task.getType()))

      { LOG.error("Unexpected event for REDUCE task " + event.getType()); task.internalError(event.getType()); }

      So, do we need to have following code in MapRetroactiveKilledTransition also just like in MapRetroactiveFailureTransition.
      if (event instanceof TaskTAttemptEvent) {
      TaskTAttemptEvent castEvent = (TaskTAttemptEvent) event;
      if (task.getState() == TaskState.SUCCEEDED &&
      !castEvent.getTaskAttemptID().equals(task.successfulAttempt))

      { // don't allow a different task attempt to override a previous // succeeded state return TaskState.SUCCEEDED; }

      }
      please check whether this is a valid case and give your suggestion.

        Attachments

        1. MAPREDUCE-4607.1.patch
          2 kB
          Bikas Saha
        2. MAPREDUCE-4607.2.patch
          18 kB
          Bikas Saha
        3. MAPREDUCE-4607.3.patch
          19 kB
          Bikas Saha
        4. MAPREDUCE-4607.patch
          19 kB
          Thomas White
        5. MAPREDUCE-4607.4.patch
          25 kB
          Bikas Saha

          Issue Links

            Activity

              People

              • Assignee:
                bikassaha Bikas Saha
                Reporter:
                bikassaha Bikas Saha
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: