Qpid
  1. Qpid
  2. QPID-1447

Broker does not handle with slow consumers effectively

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: M3
    • Fix Version/s: 0.7
    • Component/s: Java Broker
    • Labels:
      None
    • Environment:

      Any

      Description

      Please see this qpid-user thread for further context: http://markmail.org/message/vgtssb2ivyttnbsf

      The qpid java broker does not deal effectively with slow consumers. This is a combination of two issues: a MINA issue and a lack of support in qpid itself.

      The MINA issue is similar to the "fast producer" issue. In this case, the data is stored in the MINA session queues, and does not appear in the queue counts which makes it very difficult for users to track down. Each MINA session event queue needs to be a bounded list.

      The broker also needs enhancement to offer at least two options when a queue builds up:

      1) throttle the publisher. This is likely to be something that is desirable for "public" queues.

      2) disconnect the slow consumer. This will be used when dealing with private queues - typically bound to a topic exchange.

        Activity

        Robert Greig created issue -
        Hide
        Aidan Skinner added a comment -
        Show
        Aidan Skinner added a comment - See http://cwiki.apache.org/confluence/display/qpid/Broker+job+queue+limits for ways to address this.
        Martin Ritchie made changes -
        Field Original Value New Value
        Assignee Martin Ritchie [ ritchiem ]
        Martin Ritchie made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Andrew Kennedy added a comment -

        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java

        Show
        Andrew Kennedy added a comment - qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java
        Andrew Kennedy made changes -
        Attachment 0012-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12444922 ]
        Andrew Kennedy made changes -
        Attachment 0012-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12444922 ]
        Hide
        Andrew Kennedy added a comment -

        Updated due to repository changes

        Show
        Andrew Kennedy added a comment - Updated due to repository changes
        Andrew Kennedy made changes -
        Attachment 0006-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445303 ]
        Andrew Kennedy made changes -
        Attachment 0006-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445303 ]
        Andrew Kennedy made changes -
        Attachment 0015-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445340 ]
        Hide
        Marnie McCormack added a comment -

        Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.

        Show
        Marnie McCormack added a comment - Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.
        Marnie McCormack made changes -
        Fix Version/s 0.7 [ 12314455 ]
        Andrew Kennedy made changes -
        Attachment 0015-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445340 ]
        Andrew Kennedy made changes -
        Attachment 0007-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445916 ]
        Andrew Kennedy made changes -
        Attachment 0007-QPID-1447-Updated-to-use-newer-configuration-plugin.patch [ 12445916 ]
        Andrew Kennedy made changes -
        Hide
        Robbie Gemmell added a comment -

        Patch applied

        Show
        Robbie Gemmell added a comment - Patch applied
        Robbie Gemmell made changes -
        Status In Progress [ 3 ] Ready To Review [ 10006 ]
        Hide
        Robbie Gemmell added a comment -

        Resolving as child tasks are all resolved.

        Show
        Robbie Gemmell added a comment - Resolving as child tasks are all resolved.
        Robbie Gemmell made changes -
        Status Ready To Review [ 10006 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Rob Godfrey made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        542d 16h 2m 1 Martin Ritchie 07/May/10 15:20
        In Progress In Progress Reviewable Reviewable
        182d 3m 1 Robbie Gemmell 05/Nov/10 14:23
        Reviewable Reviewable Resolved Resolved
        31d 20h 43m 1 Robbie Gemmell 07/Dec/10 11:07
        Resolved Resolved Closed Closed
        1527d 8h 58m 1 Rob Godfrey 11/Feb/15 20:06

          People

          • Assignee:
            Martin Ritchie
            Reporter:
            Robert Greig
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development