[FLINK-24431] [Kinesis][EFO] EAGER registration strategy does not work when job fails over

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.12.5, 1.13.2
• Fix Version/s: 1.13.3, 1.12.8, 1.14.3, 1.15.0
• Component/s: Connectors / Kinesis
• Labels: pull-request-available
• Assignee: Rudi Kershaw
• Reporter: Danny Cranmer
• Created: 30/Sep/21 15:55
• Updated: 15/Dec/21 01:44
• Resolved: 11/Oct/21 14:26

Description

Background
The EFO Kinesis connector will register and de-register stream consumers based on the configured registration strategy. When EAGER is used, the client (usually job manager) will register the consumer and then the task managers will de-register the consumer when job stops/fails. If the job is configured to restart on fail, then the consumer will not exist and the job will continuously fail over.

Solution
The proposal is to not deregister the stream consumer when EAGER is used. The documentation should be updated to reflect this.

Issue Links

• links to: GitHub Pull Request #17417
• links to: GitHub Pull Request #17429
• links to: GitHub Pull Request #17430
• links to: GitHub Pull Request #17431