[FLINK-23183] Lost ACKs for redelivered messages in RMQSource

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.1, 1.12.4
• Fix Version/s: 1.14.0, 1.13.3, 1.12.8
• Component/s: Connectors/ RabbitMQ
• Labels: pull-request-available
• Assignee: Michał Ciesielczyk
• Reporter: Michał Ciesielczyk
• Created: 29/Jun/21 11:58
• Updated: 15/Dec/21 01:40
• Resolved: 04/Aug/21 08:20

Description

As described in the FLINK-20244, the redelivered messages are not acknowledged properly (only applicable when autoAck is disabled). When used with a prefetch count in the consumer it may even lead to stop the source to consume any more messages.

The solution (proposed in FLINK-20244) should resolve the issue. All successfully consumed RMQ messages should be acknowledged, regardless of whether the message is ignored or processed further in the pipeline. The sessionIds.add(deliveryTag) (RMQSource.java#L423) should be called before checking if the message has already been processed.

Issue Links

• is duplicated by: FLINK-20244 RMQSource does not ACK duplicated messages
• links to: GitHub Pull Request #16472