[FLINK-16573] Kinesis consumer does not properly shutdown RecordFetcher threads

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.3, 1.9.2, 1.10.0
• Fix Version/s: 1.9.3, 1.10.1, 1.11.0
• Component/s: Connectors / Kinesis
• Labels: pull-request-available
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 12/Mar/20 12:24
• Updated: 16/Mar/20 11:50
• Resolved: 16/Mar/20 11:50

Description

The threads may not shut down correctly because they do not check for the running flag in the inner loops. The threads also do not get interrupted because they are not connected to the main task thread.

These threads keep lingering around after the job has shut down:

Thread 23168: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.apache.flink.streaming.connectors.kinesis.util.RecordEmitter.emitRecords() @bci=140, line=209 (Compiled frame)
 - org.apache.flink.streaming.connectors.kinesis.util.RecordEmitter.run() @bci=18, line=177 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=748 (Compiled frame)

Issue Links

• Blocked: FLINK-16393 Kinesis consumer unnecessarily creates record emitter thread w/o source sync
• is related to: FLINK-16510 Task manager safeguard shutdown may not be reliable
• links to: GitHub Pull Request #11394