[FLINK-14300] StreamTask#invoke leaks threads if OperatorChain fails to be constructed

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.8.1, 1.8.2, 1.9.0
• Fix Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Marcos Klein
• Created: 30/Sep/19 17:40
• Updated: 11/Oct/19 12:38
• Resolved: 09/Oct/19 13:57

Description

In the StreamTask#invoke method if an exception occurs during the allocation of the operatorChain class, the exception handling fails to cleanup the threads allocated as StreamTask#recordWriters. This causes threads to leak as flink attempts to continually restart and fail for the same cause.

 

An example cause is a deserialization issue on a custom operator from a checkpoint.

 

Attached is a suggested fix for the master branch.

Issue Links

• links to: GitHub Pull Request #9857
• links to: GitHub Pull Request #9879