[MAPREDUCE-5066] JobTracker should set a timeout when calling into job.end.notification.url

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1-win, 2.0.3-alpha, 1.3.0
• Fix Version/s: 1.2.0, 1-win, 2.1.0-beta
• Target Version/s: 1.2.0, 1-win, 2.1.0-beta
• Assignee: Ivan Mitic
• Reporter: Ivan Mitic
• Created: 14/Mar/13 07:39
• Updated: 22/Aug/13 02:50
• Resolved: 20/Apr/13 19:25

Description

In current code, timeout is not specified when JobTracker (JobEndNotifier) calls into the notification URL. When the given URL points to a server that will not respond for a long time, job notifications are completely stuck (given that we have only a single thread processing all notifications). We've seen this cause noticeable delays in job execution in components that rely on job end notifications (like Oozie workflows).

I propose we introduce a configurable timeout option and set a default to a reasonably small value.

If we want, we can also introduce a configurable number of workers processing the notification queue (not sure if this is needed though at this point).

I will prepare a patch soon. Please comment back.

Issue Links

• is duplicated by: MAPREDUCE-4935 Support timeout limitation to MRv1 job end notifications