1) Configure resource profile in server side or client side:
Currently resource profile can be only configured centrally:
A given resource profile has a the same meaning in the cluster. It won’t change when we run different apps in different configurations. A job can run under Amazon’s G2.8X can also run on YARN with G2.8X profile. A side benefit is YARN scheduler can potentially do better bin packing.
Hard for applications to add their own resource profiles.
2) Do we really need mandatory resource profiles such as minimum/maximum/default?
3) Should we send resource profile name inside ResourceRequest, or should client/AM translate it to resource and set it to the existing resource fields?
4) Related to above, should we allow resource overrides or client/AM should send final resource to RM?