Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1542

Incorrect task/tip being scheduled (looks like speculative execution)

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.14.0
    • 0.14.0
    • None
    • None

    Description

      The change in HADOOP-1440 broke map/reduce by breaking the assumption that Task.getPartition() corresponded to the JobInProgress.map[] order.

      Currently JobInProgress.findNewTask uses Task.getPartition as the index of the map to run. This can be a completely different tip, which will cause incorrect tasks to be run, including duplicates of tasks that are already running.

      Attachments

        1. jobtracker.log
          117 kB
          Nigel Daley
        2. namenode.log
          1.20 MB
          Nigel Daley

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            omalley Owen O'Malley
            nidaley Nigel Daley
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment