Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-5313

ActiveMq consumer intermittently hanging after reconnect

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.9.0
    • Fix Version/s: 5.9.1
    • Component/s: JMS client
    • Labels:
      None
    • Environment:

      AIX 6.1, AIX 7.1, IBM Java 6, IBM Java 7

      Description

      I have run into an intermittent problem with ActiveMQ V5.9.0. The intermittent was tracked down to being a connection error that does not get properly recovered.

      I have recreated the problem in a very small example by creating a Camel
      consumer route that retrieves messages from a JMS queue and writes the text message to a log.

      from(TEST_QUEUE).routeId(TEST_QUEUE).convertBodyTo(String.class)
      .log(LoggingLevel.INFO, loggerName, "Request Received ");

      The connection string used is
      failover:(tcp://serverd05.company.com:26093?keepAlive=true)?jms.prefetchPolicy.all=0

      Please note that polling is being used when the prefetch size is set to 0.

      I then use Hawtio on the ActiveMQ broker to add some text messages to the queue. It works fine.

      I then stop the Client Connector on the ActiveMQ broker side to simulate a
      broken connection on the consumer side. (The broker must be remote to the consumer or the error will not occur.) The client logs show that an
      EOFException is caught and the connection is reestablished. The connection also appears on the ActiveMQ broker. it only looks good though. Sending new text messages to the queue will not be processed. They will just sit there. There are no errors or warnings logged on either the consumer or broker servers. Restarting the consumer will cause the messages to get processed. The reconnection only intermittently fails. I find is much more like to occur if 10 minutes pass from the previous message being processed.

      I turned on trace=yes in the connection string and found the root cause is the PullMessage commands are occasionally not being issued after a reconnect. On cases where it works, the log shows that the PullMessage commands do reinstate.

      I am wondering if this might be a race condition, as this problem only showed up when the consumer was on fast servers

        Attachments

        1. fail.log
          33 kB
          Wayne Irwin

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              wayn23 Wayne Irwin
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: