[YARN-10449] Flexing doesn't consider containers which were stopped

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Bilwa S T
• Reporter: Bilwa S T
• Created: 28/Sep/20 13:32
• Updated: 29/Jan/21 12:37

Description

we have use case where if worker is idle for some period of time then user would want to shutdown the worker to release resource and request more workers when load is more.

In case of ON_FAILURE retry policy if user gracefully shutdown worker its exit status will be 0 so container wont be relaunched. In this case
if user try to flex up then it currently doesn't consider stopped containers which is not correct.
i could think of two possible solutions:
1. Consider deducting succeeded containers from Number Of Containers and then clear succeeded component if flex up/down is done.
2. Consider updating number of containers when stopped

Issue Links

• is related to: YARN-8080 YARN native service should support component restart policy