[BEAM-7348] Option to expire SDK worker environments

• Type: Improvement
• Status: Triage Needed
• Priority: P2
• Resolution: Implemented
• Affects Version/s: 2.12.0
• Fix Version/s: 2.14.0
• Component/s: runner-core
• Labels: portability portability-flink
• Assignee: Thomas Weise
• Reporter: Thomas Weise
• Created: 17/May/19 13:11
• Updated: 13/Apr/23 10:55
• Resolved: 23/May/19 17:55

Description

We discovered that Python SDK workers are susceptible to memory leaks that are quite hard to identify and/or fix. This becomes an issue in streaming pipelines, where the workers run "forever". It would be good if the user has an option to recycle the workers when there is no other practical way to address (slow) resource leaks.

Issue Links

• links to: GitHub Pull Request #8599
• links to: GitHub Pull Request #8714