Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
  3. FLINK-14405

Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.

    XMLWordPrintableJSON

    Details

      Description

      According to FLIP-49, a task can consume the following TaskExecutor resources, which will also be the fields of ResourceProfile.

      • CPU
      • Task Heap Memory
      • Task Off-Heap Memory
      • On-Heap Managed Memory
      • Off-Heap Managed Memory
      • Shuffle Memory
      • Extended Resources

      All the resource requirements should be specified by the operators, except Shuffle Memory which should be decided by JobMaster according to the topology of the physical graph. Therefore, a ResourceSpec should have the following fields.

      • CPU
      • Task Heap Memory
      • Task Off-Heap Memory
      • On-Heap Managed Memory
      • Off-Heap Managed Memory
      • Extended Resources

        Attachments

          Activity

            People

            • Assignee:
              xintongsong Xintong Song
              Reporter:
              xintongsong Xintong Song
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m