[CAMEL-20660] camel-azure-servicebus: Consumer fails to acknowledge messages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.4.0, 4.4.1, 4.5.0
• Fix Version/s: 4.7.0
• Component/s: camel-azure
• Estimated Complexity: Unknown
• Assignee: Dylan Piergies
• Reporter: Dylan Piergies
• Created: 09/Apr/24 12:45
• Updated: 05/Jun/24 09:24
• Resolved: 05/Jun/24 09:24

Description

We have observed issues with the Service Bus consumer from Camel 4.4+ where consumed messages are not acknowledged/completed correctly and landing in the dead-letter queue, despite the Exchange having successfully delivered the message to its destination.

Our routes all follow the general form:

from(azureServicebus(...))
  // ...
  .to(https(...))
  .log("Message delivered to...");

In our logs, we are seeing the message from the final log EIP and we can confirm that the message has been delivered to the destination service, but this is often followed by a log message from the com.azure.messaging.servicebus.ServiceBusReceiverAsyncClient logger:

Cannot perform operation 'completed' on a disposed receiver.

We then see many of these messages arriving in the DLQ, once the retry count is exceeded.

The issue is difficult to create a reproduction for: it is intermittent and occurs most frequently during a spike in message volumes.

The issue disappears after downgrading Camel to 4.3.0.

Whilst we do not know the root cause for sure, we suspect this may be a defect of CAMEL-19262, occurring if the client is closed and recreated while an Exchange is in flight.

Related Zulip chat discussion: https://camel.zulipchat.com/#narrow/stream/257298-camel/topic/Azure.20Service.20Bus.20Consumer.20failing.20to.20acknowledge.20messages

Issue Links

• is related to: CAMEL-20693 camel-azure-servicebus: Service Bus Listener Not Stopping After Stopping Route in Latest Camel Version(4.4.0)
• relates to: CAMEL-20266 camel-azure-eventbus - Use high-level client (ServiceBusProcessorClient)
• links to: GitHub Pull Request #14314