Details
Description
It will be useful to support Application Timeout in YARN. Some use cases are not worried about the output of the applications if the application is not completed in a specific time.
Background:
The requirement is to show the CDR statistics of last few minutes, say for every 5 minutes. The same Job will run continuously with different dataset.
So one job will be started in every 5 minutes. The estimate time for this task is 2 minutes or lesser time.
If the application is not completing in the given time the output is not useful.
Proposal
So idea is to support application timeout, with which timeout parameter is given while submitting the job.
Here, user is expecting to finish (complete or kill) the application in the given time.
One option for us is to move this logic to Application client (who submit the job).
But it will be nice if it can be generic logic and can make more robust.
Kindly provide your suggestions/opinion on this feature. If it sounds good, i will update the design doc and prototype patch
Attachments
Attachments
Issue Links
- duplicates
-
YARN-8573 Add Maximum lifetime for YARN applications
- Resolved
- is related to
-
YARN-2266 Add an application timeout service in RM to kill applications which are not getting resources
- Open
-
YARN-5692 Support more application timeout types
- Open
- relates to
-
AMBARI-24387 Support YARN Application timeout feature in Ambari Capacity Scheduler View
- Resolved
-
YARN-2487 Need to support timeout of AM When no containers are assigned to it for a defined period
- Resolved
-
MAPREDUCE-6490 Facilitate the application life time out in mapreduce applications.
- Open