[FLINK-13983] Launch task executor with new memory calculation logics

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Component/s: Runtime / Task
• Labels: pull-request-available
• Release Note: The usage of environment variable FLINK_TM_HEAP is deprecated. Please, use Flink option "taskmanager.memory.total-process.size" in yaml configuration file to set the total task executor memory. See FLIP-49 for more details.
• Assignee: Xintong Song
• Reporter: Xintong Song
• Created: 06/Sep/19 02:54
• Updated: 03/Dec/19 15:38
• Resolved: 03/Dec/19 15:37

Description

• Invoke data structures and utilities introduced in Step 2 to generate JVM parameters and dynamic configurations for launching new task executors.
  • In startup scripts
  • In resource managers
• Task executor uses data structures and utilities introduced in Step 2 to set memory pool sizes and slot resource profiles.
  • MemoryManager
  • ShuffleEnvironment
  • TaskSlotTable

Implement this step as separate code paths only for the new mode.

Issue Links

• is duplicated by: FLINK-14404 the jvm heap size from calculateTaskManagerHeapSizeMB in config.sh is different from TaskManagerServices.calculateHeapSizeMB
• links to: GitHub Pull Request #9801