[FLINK-22479] [Kinesis][Consumer] Potential lock-up under error condition

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.12.0, 1.12.1, 1.12.2, 1.13.0, 1.12.3
• Fix Version/s: 1.14.0, 1.13.1, 1.12.4
• Component/s: Connectors / Kinesis
• Labels: pull-request-available
• Assignee: Danny Cranmer
• Reporter: Danny Cranmer
• Created: 26/Apr/21 12:59
• Updated: 28/May/21 11:06
• Resolved: 10/May/21 13:34

Description

Background
This connector has been forked by AWS for use on KDA with Flink 1.11. Bugs have been encountered:

• Under high backpressure scenarios
• When an error is thrown during tear down

Scope
Pull in the following fixes from AWS fork:

• Fix issue where KinesisDataFetcher.shutdownFetcher() hangs (issue, pull request)

• Log error when shutting down Kinesis Data Fetcher (issue, pull request)

• Treating TimeoutException as Recoverable Exception (issue, pull request)

• Add time-out for acquiring subscription and passing events from network to source thread to prevent deadlock (pull request)

Issue Links

• links to: GitHub Pull Request #15770
• links to: GitHub Pull Request #15773
• links to: GitHub Pull Request #15774