Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Incomplete
-
5.5.0
-
None
-
None
-
Linux AS5 (CentOS release 5.5),
Java: SUNJAVA2 1.6.0r14a
Description
We have an issue with AMQ performance. In our test program, we run two amq nodes in a network cluster. Both AMQ client and server/worker consist of producers and consumers. When client makes a connection to an AMQ node, it will send messages to three queues and meanwhile listen to 2 topics and one temporary queue. In AMQ worker/server side, it will listen to three queues and send messages back to corresponding two topics and one temp queue. During our test, We first had 10 servers/workers running at the same node as one AMQ broker was running, then started client program remotely by making 50 connections first, then repeated the 50 connections again and again. We found when the connection number reached above 650, AMQ slowed down significantly, as when we tried to send a text message through Jconsole (which run remotely also); we couldn't get a response from Jconsole with a returned messageID. Any message sent during that time was not enqueued in time. Looks like there was a blocking to the producer side, but we had producer flow control turned off. The heap memory usage looked fine to us during that time (well below 3G we assigned for JVM). But we did notice that we even couldn't do a thread dump during that time, as it kept failing to attach the process. It took some time (from 10 seconds to 15 minutes or more) for the AMQ to come back and response to the messages sent, and by then we could do thread dump without a problem.
In amq configure side, I have followed suggestion by Dave Stanley http://www.pepperdust.org/?p=150. Not sure why we still have such a "hung" problem? I'll attach the test case program with our amq configure file latter.