[FLINK-27303] Flink Operator will create a large amount of temp log config files

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: kubernetes-operator-1.0.0
• Fix Version/s: kubernetes-operator-1.0.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Gyula Fora
• Reporter: Gyula Fora
• Created: 19/Apr/22 07:12
• Updated: 04/May/22 14:48
• Resolved: 04/May/22 14:48

Description

Now we use the configbuilder in multiple different places to generate the effective config including observer, reconciler, validator etc.

The effective config gerenration logic also creates temporary log config files (if spec logConfiguration is set) which would lead to 3-4 files generated in every reconcile loop for a given job. These files are not cleaned up until the operator restarts leading to a large amount of files.

I believe we should change the config generation logic and only apply the logconfig generation logic right before flink cluster submission as that is the only thing affected by it.

Issue Links

• links to: GitHub Pull Request #177
• links to: GitHub Pull Request #191