[FLINK-15848] Support both fixed allocator and dynamic allocator in flink

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: liupengcheng
• Created: 03/Feb/20 02:28
• Updated: 19/Nov/21 10:39

Description

Currently, we removed static allocator and only support dynamic allocation in flink1.10, however, this allocator still has some drawbacks:

1. Can not allocate resources in a range, which means the resource usage is not under control, this has very bad effect in a shared resource cluster(e.g. Yarn), one big query or job may occupy all the resources and  cause other jobs block.
2. Not support static resource allocation. That means we can hardly do benchmark testing across engines(e.g. Spark). Also, in resource shared cluster(e.g. Yarn) that support over-allocation, it's hard to align the resources usage.

As discussed in FLINK-12362 ,  we should support both fixed allocator and dynamic allocator(dynamically allocating in a range) in flink.