-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1-win, 2.0.3-alpha, 1.3.0
-
Fix Version/s: 1.2.0, 1-win, 2.1.0-beta
-
Component/s: None
-
Labels:None
-
Target Version/s:
In current code, timeout is not specified when JobTracker (JobEndNotifier) calls into the notification URL. When the given URL points to a server that will not respond for a long time, job notifications are completely stuck (given that we have only a single thread processing all notifications). We've seen this cause noticeable delays in job execution in components that rely on job end notifications (like Oozie workflows).
I propose we introduce a configurable timeout option and set a default to a reasonably small value.
If we want, we can also introduce a configurable number of workers processing the notification queue (not sure if this is needed though at this point).
I will prepare a patch soon. Please comment back.
- is duplicated by
-
MAPREDUCE-4935 Support timeout limitation to MRv1 job end notifications
-
- Resolved
-