Yarn itself supports rack/node aware scheduling for AMs; however, there currently are two problems:
- To specify hard or soft rack/node requests, you have to specify more than one ResourceRequest. For example, if you want to schedule an AM only on "rackA", you have to create two ResourceRequest, like this:
The problem is that the Yarn API doesn't actually allow you to specify more than one ResourceRequest in the ApplicationSubmissionContext. The current behavior is to either build one from getResource or directly from getAMContainerResourceRequest, depending on if getAMContainerResourceRequest is null or not. We'll need to add a third method, say getAMContainerResourceRequests, which takes a list of ResourceRequest so that clients can specify the multiple resource requests.
- There are some places where things are hardcoded to overwrite what the client specifies. These are pretty straightforward to fix.