Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5801

JobTracker should refresh the hosts list upon recovery

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • None
    • None
    • Reviewed

    Description

      If the hosts file is changes across restart then it should be refreshed upon recovery so that the excluded hosts are lost and the maps are re-executed.
      The mapred-hosts list can change across restarts. Once the jobtracker recovers, it detects all the tasktracker there are there in the history. If the hosts list is changed, then the jobtracker will still have the tasktracker (data) internally but will disallow the tracker when it contacts. As a result, the jobtracker will have to wait for the tracker to timeout in order to re-execute the tasks. This patch simply refreshes the node list upon recovery so that the invalid trackers are lost immediately.

      Attachments

        1. 5801-0.20.patch
          5 kB
          Robert Chansler
        2. HADOOP-5801-v1.2.patch
          5 kB
          Amar Kamat

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amar_kamat Amar Kamat
            amar_kamat Amar Kamat
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment