Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-39

Task tracker should wait for the process to exit before declaring the task successful or failed.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently when a task declares it is done, the status in the task tracker is changed immediately. Instead it should wait for the subprocess to actually be done before it moves to one of the final states. This lead to a race condition where the task was still generating log data after the job tracker had reported the task as done.

        Activity

        Hide
        Harsh J added a comment -

        This'd be complicated with JVM reuse. Besides, if a task reports itself done, its done?

        Resolving but reopen if am wrong.

        Show
        Harsh J added a comment - This'd be complicated with JVM reuse. Besides, if a task reports itself done, its done? Resolving but reopen if am wrong.
        Hide
        Devaraj Das added a comment -

        Ignore my previous comment. This doesn't actually hold true in the case of jvm reuse.

        Show
        Devaraj Das added a comment - Ignore my previous comment. This doesn't actually hold true in the case of jvm reuse.
        Hide
        Devaraj Das added a comment -

        I forgot to take care of this issue in the JVM reuse patch too. In that, the JVM is explicitly killed if it is found to be idle. But it might sometimes have an unfortunate side effect to do with logs not fully flushed yet.

        Show
        Devaraj Das added a comment - I forgot to take care of this issue in the JVM reuse patch too. In that, the JVM is explicitly killed if it is found to be idle. But it might sometimes have an unfortunate side effect to do with logs not fully flushed yet.
        Hide
        Doug Cutting added a comment -

        If the task has more to do, shouldn't it wait to declare itself done? Or perhaps we should introduce a new task status, "exiting" or somesuch, which means that the output is complete but the jvm hasn't exited and hence the slot is not yet free. If a task hangs after completing it's output but before exiting, then the tasktracker should still kill it, but the jobtracker should not reschedule it, right?

        Show
        Doug Cutting added a comment - If the task has more to do, shouldn't it wait to declare itself done? Or perhaps we should introduce a new task status, "exiting" or somesuch, which means that the output is complete but the jvm hasn't exited and hence the slot is not yet free. If a task hangs after completing it's output but before exiting, then the tasktracker should still kill it, but the jobtracker should not reschedule it, right?

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development