[FLINK-15145] Tune default values for FLIP-49 TM memory configurations with real production jobs.

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Done
• Fix Version/s: 1.10.0
• Component/s: Runtime / Configuration
• Labels: pull-request-available
• Assignee: Xintong Song
• Reporter: Xintong Song
• Created: 09/Dec/19 08:48
• Updated: 27/Apr/21 18:05
• Resolved: 16/Jan/20 08:03

Description

In release 1.10, with FLIP-49, we introduced significant changes to the TaskExecutor memory model and it's related configuration options / logics.

Since the model and configuration logics are changed, it is reasonable that we also change the default configuration values. Currently, the default values are set with the gut feelings and experiences from e2e tests. It would be good that we try and tune the configurations with some real production jobs, of various scales if possible, before exposing the configurations in the release.

Issue Links

• blocks: FLINK-15143 Create document for FLIP-49 TM memory model and configuration guide
• relates to: FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
• links to: Dev mail list discussion
• links to: GitHub Pull Request #10860