Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1542

Incorrect task/tip being scheduled (looks like speculative execution)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      The change in HADOOP-1440 broke map/reduce by breaking the assumption that Task.getPartition() corresponded to the JobInProgress.map[] order.

      Currently JobInProgress.findNewTask uses Task.getPartition as the index of the map to run. This can be a completely different tip, which will cause incorrect tasks to be run, including duplicates of tasks that are already running.

        Attachments

        1. namenode.log
          1.20 MB
          Nigel Daley
        2. jobtracker.log
          117 kB
          Nigel Daley

          Activity

            People

            • Assignee:
              omalley Owen O'Malley
              Reporter:
              nidaley Nigel Daley
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: