[FLINK-3642] Disentangle ExecutionConfig

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: API / DataSet, API / DataStream
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 21/Mar/16 13:59
• Updated: 21/Sep/22 09:11

Description

Initially, the ExecutionConfig started out being a configuration to configure the behaviour of the system with respect to the associated job. As such it stored information about the restart strategy, registered types and the parallelism of the job. However, it happened that the ExecutionConfig has become more of an easy entry-point to pass information into the system. As such, the user can now set arbitrary information as part of the GlobalJobParameters in the ExecutionConfig which is piped to all kinds of different locations in the system, e.g. the serializers, JM, ExecutionGraph, TM, etc.

This mixture of user code classes with system parameters makes it really cumbersome to send system information around, because you always need a user code class loader to deserialize it. Furthermore, there are different means how the ExecutionConfig is passed to the system. One is giving it to the Serializers created in the JavaAPIPostPass and another is giving it directly to the JobGraph, for example. The problem is that the ExecutionConfig contains information which is required at different stages of a program execution.

I think it would be beneficial to disentangle the ExecutionConfig a little bit along the lines of the different concerns for which the ExecutionConfig is used currently.

Issue Links

• relates to: FLINK-29379 Back (most of the) ExecutionConfig and CheckpointConfig by Configuration