Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15848

Support both fixed allocator and dynamic allocator in flink

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Component/s: Runtime / Task
    • Labels:
      None

      Description

      Currently, we removed static allocator and only support dynamic allocation in flink1.10, however, this allocator still has some drawbacks:

      1. Can not allocate resources in a range, which means the resource usage is not under control, this has very bad effect in a shared resource cluster(e.g. Yarn), one big query or job may occupy all the resources and  cause other jobs block.
      2. Not support static resource allocation. That means we can hardly do benchmark testing across engines(e.g. Spark). Also, in resource shared cluster(e.g. Yarn) that support over-allocation, it's hard to align the resources usage.

      As discussed in FLINK-12362 ,  we should support both fixed allocator and dynamic allocator(dynamically allocating in a range) in flink.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              liupengcheng liupengcheng
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: