Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
  3. FLINK-14400

Shrink the scope of MemoryManager from TaskExecutor to slot

    XMLWordPrintableJSON

Details

    Description

      MemoryManager currently manages the memory bookkeeping for all slots/tasks inside one TaskExecutor. For better abstraction and isolation of slots, we can shrink its scope and make it per slot. The memory limits are fixed now per slot at the moment of slot creation. All operators, sharing the slot, will get their fixed fractional limits.

      In future, we might make it possible for operators to over-allocate beyond their fraction limit if there is some available free memory in the slot but it should be possible to reclaim the over-allocated memory at any time if other operator decides to claim its fair share within its limit.

      Attachments

        Issue Links

          Activity

            People

              azagrebin Andrey Zagrebin
              azagrebin Andrey Zagrebin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m