[AMQ-3966] Messages get stuck in queue

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Abandoned
• Affects Version/s: 5.5.0, 5.5.1, 5.6.0
• Component/s: Broker

Environment

java version "1.6.0_32"
Java(TM) SE Runtime Environment (build 1.6.0_32-b05)
Java HotSpot(TM) 64-Bit Server VM (build 20.7-b02, mixed mode)

Linux ****** 2.6.18-308.4.1.el5 #1 SMP Tue Apr 17 17:08:00 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

CentOS release 5.8 (Final)
Kernel \r on an \m

• Assignee: Unassigned
• Reporter: Robin Kåveland
• Created: 09/Aug/12 08:37
• Updated: 13/Mar/22 14:08
• Resolved: 13/Mar/22 14:08

Description

The broker is using the jdbc-persistence-adapter and persists to an oracle database. We have so far been unable to reproduce this issue reliably, but it appears both in our testing and production environments with some regularity.

The issue we are having is superficially very similar to this description. Retrieving the queue message count with jmx or inspecting the activemq database with a

SELECT COUNT(*) FROM ACTIVEMQ_MSGS WHERE CONTAINER LIKE '%DeadLetterQueue%'

results in a number that is larger than the amount of messages which can actually be retrieved using a QueueBrowser or a QueueViewMBean. This difference grows over time, as no messages that are sent to the queue after this problem occurs are retrievable. We have also observed this problem happening on queues with normal JMS consumers, where the queue will grow without the consumer receiving any messages.

A broker restart can sometimes resolve this issue. Our current workaround is to use a program that fetches the rows for the queue from the database, unmarshals them and sends them to a temporary queue. The queue with the stuck messages does not work (does not deliver any messages to consumers or browsers) until after it has been purged.