[YARN-110] AM releases too many containers due to the protocol

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: resourcemanager, scheduler
• Assignee: Arun C Murthy
• Reporter: Arun C Murthy
• Created: 15/Jul/11 22:39
• Updated: 07/Jun/16 13:47

Description

• AM sends request asking 4 containers on host H1.
• Asynchronously, host H1 reaches RM and gets assigned 4 containers. RM at this point, sets the value against H1 to
  zero in its aggregate request-table for all apps.
• In the mean-while AM gets to need 3 more containers, so a total of 7 including the 4 from previous request.
• Today, AM sends the absolute number of 7 against H1 to RM as part of its request table.
• RM seems to be overriding its earlier value of zero against H1 to 7 against H1. And thus allocating 7 more
  containers.
• AM already gets 4 in this scheduling iteration, but gets 7 more, a total of 11 instead of the required 7.

Issue Links

• is related to: YARN-1902 Allocation of too many containers when a second request is done with the same resource capability
• is related to: MAPREDUCE-4671 AM does not tell the RM about container requests that are no longer needed
• relates to: YARN-4879 Enhance Allocate Protocol to Identify Requests Explicitly