ActiveMQ
  1. ActiveMQ
  2. AMQ-3738

Cursors not dealing with topic memoryLimit overflow

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 5.5.1
    • Fix Version/s: None
    • Component/s: Broker
    • Labels:
      None
    • Environment:

      OSX, Apple Java Runtime 1.6.0_26

      Description

      I have defined a topic with a memoryLimit of 1mb using a fileDurableSubscriberCursor. I would expect that sending messages to a durable subscriber through that topic would result in more than 1mb worth of messages being able to be sent. Instead producer flow control kicks in at the 1mb limit.

      To test this I am using the Maven activemq-perf plugin with a single consumer and producer:
      $ mvn activemq-perf:producer -Durl=tcp://localhost:61616 -Dproducer.destName=topic://orders
      $ mvn activemq-perf:consumer -Durl=tcp://localhost:61616 -Dconsumer.durable=true -Dconsumer.destName=topic://orders -Dconsumer.recvDelay=100

      This does not match the behaviour described in http://activemq.apache.org/my-producer-blocks.html or http://activemq.apache.org/message-cursors.html
      Going by the docs, I would expect messages to get paged out to disk until the systemUsage/tempUsage limit fills up.

      This same behaviour happens using the default cursor configuration.

      Attached are the activemq-perf pom and activemq.xml.

      1. pom.xml
        2 kB
        Jakub Korab
      2. activemq-cursors.xml
        2 kB
        Jakub Korab

        Activity

        Jakub Korab created issue -
        Jakub Korab made changes -
        Field Original Value New Value
        Attachment pom.xml [ 12515934 ]
        Jakub Korab made changes -
        Attachment activemq-cursors.xml [ 12515935 ]
        Hide
        Torsten Mielke added a comment -

        IIRC then producer flow control will kick in. Paging out to disk only happens when 100% of the memoryLimit are used. At that same time producer flow control will kick in as well because of the 100% limit being reached. Paging to disk will only occur if producer flow control is disabled.
        But I presume the limit at which paging to disk happens may be configurable.

        Show
        Torsten Mielke added a comment - IIRC then producer flow control will kick in. Paging out to disk only happens when 100% of the memoryLimit are used. At that same time producer flow control will kick in as well because of the 100% limit being reached. Paging to disk will only occur if producer flow control is disabled. But I presume the limit at which paging to disk happens may be configurable.
        Hide
        Jakub Korab added a comment -

        I ran the same thing with PFC turned off. VisualVM shows MemoryPercentUsage on the topic steadily rise to 100%, then drop to 0%. No further messages are dequeued or enqueued.

        Only difference to earlier attached config was:

        <policyEntry topic=">" producerFlowControl="false" memoryLimit="1mb">

        Show
        Jakub Korab added a comment - I ran the same thing with PFC turned off. VisualVM shows MemoryPercentUsage on the topic steadily rise to 100%, then drop to 0%. No further messages are dequeued or enqueued. Only difference to earlier attached config was: <policyEntry topic=">" producerFlowControl="false" memoryLimit="1mb">
        Hide
        Mark Anderson added a comment - - edited

        I modified the test case testFillTempAndConsume in http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/bugs/TempStorageBlockedBrokerTest.java?view=markup&pathrev=1233367 to use a topic instead of a queue and it failed.

        Is this the same issue?

        Show
        Mark Anderson added a comment - - edited I modified the test case testFillTempAndConsume in http://svn.apache.org/viewvc/activemq/trunk/activemq-core/src/test/java/org/apache/activemq/bugs/TempStorageBlockedBrokerTest.java?view=markup&pathrev=1233367 to use a topic instead of a queue and it failed. Is this the same issue?
        Hide
        Mark Anderson added a comment -

        From posting on the mailing list I found that I needed to add setting the topic prefetch policy on my connection factories to enable temp storage with topics and non-persistent messages so my previous comment was incorrect.

        Show
        Mark Anderson added a comment - From posting on the mailing list I found that I needed to add setting the topic prefetch policy on my connection factories to enable temp storage with topics and non-persistent messages so my previous comment was incorrect.
        Hide
        Claus Ibsen added a comment -

        Whats the status of this ticket?

        Show
        Claus Ibsen added a comment - Whats the status of this ticket?

          People

          • Assignee:
            Unassigned
            Reporter:
            Jakub Korab
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development