ActiveMQ
  1. ActiveMQ
  2. AMQ-2745

Deadlock or Performance Bottleneck when reading messages with Correlation

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 5.3.1, 5.3.2, 5.4.0, 5.4.1
    • Fix Version/s: NEEDS_REVIEW
    • Component/s: Broker
    • Labels:
      None
    • Environment:

      Java 64-bit, Windows 2008 Server, Centos 5 64bit, Ubuntu 110 64-bit, OS X 10.5, 10.6

      Description

      We have a situation where we are posting messages to a queue with two different correlation ids specifically intended to reach two different clients who subscribe with message selectors for the appropriate correlation. The clients are reading with message listeners. When one client stops reading the expected behavior, and the behavior we saw on 5.3.0, is that the messages with the correlation for the stopped client will backup on the queue and will not effect the performance of the second client who is still reading the messages with the other correlation. With our memory config messages can backup into the hundreds of thousands before noticing any performance impact on the active client.

      However this is not the case in 5.3.1 or 5.3.2. With 5.3.1 once enough messages backup for the stopped client, suddenly the active client's performance drops drastically 20 ms reads to 30,000ms reads. We will see this within a few hundred messages. I believe there is some kind of deadlock, or buffering bottleneck that was introduced on the client side.

      1. activemq.xml
        5 kB
        Brad Willard
      2. PutMessages.java
        2 kB
        Brad Willard
      3. ReadMessages.java
        2 kB
        Brad Willard
      4. activemq.xml
        5 kB
        Brad Willard

        Issue Links

          Activity

          Brad Willard created issue -
          Rob Davies made changes -
          Field Original Value New Value
          Fix Version/s 5.4.1 [ 12332 ]
          Brad Willard made changes -
          Affects Version/s 5.3.2 [ 12310 ]
          Fix Version/s 5.4.1 [ 12332 ]
          Priority Minor [ 4 ] Major [ 3 ]
          Description We have a situation where we are posting messages to a queue with two different correlation ids specifically intended to reach two different clients who subscribe with message selectors for the appropriate correlation. The clients are reading with message listeners. When one client stops reading the expected behavior, and the behavior we saw on 5.3.0, is that the messages with the correlation for the stopped client will backup on the queue and will not effect the performance of the second client who is still reading the messages with the other correlation. With our memory config messages can backup into the hundreds of thousands before noticing any performance impact on the active client.

          However this is not the case in 5.3.1. With 5.3.1 once enough messages backup for the stopped client, suddenly the active client's performance drops drastically 20 ms reads to 30,000ms reads. We will see this within a few hundred messages. I believe there is some kind of deadlock, or buffering bottleneck that was introduced on the client side.
          We have a situation where we are posting messages to a queue with two different correlation ids specifically intended to reach two different clients who subscribe with message selectors for the appropriate correlation. The clients are reading with message listeners. When one client stops reading the expected behavior, and the behavior we saw on 5.3.0, is that the messages with the correlation for the stopped client will backup on the queue and will not effect the performance of the second client who is still reading the messages with the other correlation. With our memory config messages can backup into the hundreds of thousands before noticing any performance impact on the active client.

          However this is not the case in 5.3.1 or 5.3.2. With 5.3.1 once enough messages backup for the stopped client, suddenly the active client's performance drops drastically 20 ms reads to 30,000ms reads. We will see this within a few hundred messages. I believe there is some kind of deadlock, or buffering bottleneck that was introduced on the client side.
          Environment Java 64-bit, Windows 2008 Server Java 64-bit, Windows 2008 Server, Centos 5 64bit, Ubuntu 110 64-bit
          Brad Willard made changes -
          Attachment activemq.xml [ 19454 ]
          Brad Willard made changes -
          Attachment PutMessages.java [ 19455 ]
          Attachment ReadMessages.java [ 19456 ]
          Brad Willard made changes -
          Affects Version/s 5.4.0 [ 12110 ]
          Brad Willard made changes -
          Fix Version/s NEEDS_REVIEWED [ 12186 ]
          Brad Willard made changes -
          Attachment activemq.xml [ 19634 ]
          Gary Tully made changes -
          Link This issue duplicates AMQ-2217 [ AMQ-2217 ]
          Jeff Turner made changes -
          Project Import Fri Nov 26 22:32:02 EST 2010 [ 1290828722158 ]
          Brad Willard made changes -
          Affects Version/s 5.4.1 [ 12315624 ]
          Environment Java 64-bit, Windows 2008 Server, Centos 5 64bit, Ubuntu 110 64-bit Java 64-bit, Windows 2008 Server, Centos 5 64bit, Ubuntu 110 64-bit, OS X 10.5, 10.6
          Timothy Bish made changes -
          Link This issue is part of AMQ-2217 [ AMQ-2217 ]
          Timothy Bish made changes -
          Status Open [ 1 ] Closed [ 6 ]
          Resolution Duplicate [ 3 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Brad Willard
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development