[FLINK-14536] Make clear the way to aggregate specified cpuCores resources

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Zhu Zhu
• Reporter: Zhu Zhu
• Created: 27/Oct/19 16:22
• Updated: 03/Nov/19 20:32
• Resolved: 03/Nov/19 20:32

Description

I'm raising this question because I find cpuCores in ResourceSpec#merge are aggregated with max(), while in ResourceProfile#merge it is sum().

This means that when calculating resources of a vertex from within operators, the cpuCores is the max value. While it is a sum(or subtraction in ResourceProfile#subtract) when dealing with shared slot bookkeeping and related checks.
This is confusing to me, especially when I'm considering how to generate a shared slot resource spec merged from all vertices in it(see FLINK-14314).

I'm not pretty sure if we already have a concise definition for it?
If there is, we need to respect it and change ResourceSpec or ResourceProfile accordingly.
If not, we need to decide it first before we can move on with fine grained resources.

Need to mention that if we take the max() way, we need to re-consider how we can conduct a correct ResourceProfile#subtract regarding cpuCores, since there is not a clear way to reverse a max() computation.

cc trohrmann xintongsong azagrebin

Issue Links

• links to: GitHub Pull Request #10072