Details
-
Improvement
-
Status: Open
-
Not a Priority
-
Resolution: Unresolved
-
1.10.0
-
None
Description
Currently, we removed static allocator and only support dynamic allocation in flink1.10, however, this allocator still has some drawbacks:
- Can not allocate resources in a range, which means the resource usage is not under control, this has very bad effect in a shared resource cluster(e.g. Yarn), one big query or job may occupy all the resources and cause other jobs block.
- Not support static resource allocation. That means we can hardly do benchmark testing across engines(e.g. Spark). Also, in resource shared cluster(e.g. Yarn) that support over-allocation, it's hard to align the resources usage.
As discussed in FLINK-12362 , we should support both fixed allocator and dynamic allocator(dynamically allocating in a range) in flink.