[BEAM-8618] Tear down unused DoFns periodically in Python SDK harness

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Missing
• Component/s: sdk-py-harness
• Labels: stale-P2
• Assignee: sunjincheng
• Reporter: sunjincheng
• Created: 12/Nov/19 13:40
• Updated: 13/Jan/22 21:52
• Resolved: 15/May/21 17:55

Description

Per the discussion in the ML, detail can be found [1], the teardown of DoFns should be supported in the portability framework. It happens at two places:
1) Upon the control service termination
2) Tear down the unused DoFns periodically

The aim of this JIRA is to add support for tear down the unused DoFns periodically in Python SDK harness.
[1] https://lists.apache.org/thread.html/0c4a4cf83cf2e35c3dfeb9d906e26cd82d3820968ba6f862f91739e4@%3Cdev.beam.apache.org%3E

Issue Links

• Is contained by: BEAM-8325 SdkWorker should be stopped in SdkHarness
• links to: GitHub Pull Request #10655