Qpid
  1. Qpid
  2. QPID-1447

Broker does not handle with slow consumers effectively

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: M3
    • Fix Version/s: 0.7
    • Component/s: Java Broker
    • Labels:
      None
    • Environment:

      Any

      Description

      Please see this qpid-user thread for further context: http://markmail.org/message/vgtssb2ivyttnbsf

      The qpid java broker does not deal effectively with slow consumers. This is a combination of two issues: a MINA issue and a lack of support in qpid itself.

      The MINA issue is similar to the "fast producer" issue. In this case, the data is stored in the MINA session queues, and does not appear in the queue counts which makes it very difficult for users to track down. Each MINA session event queue needs to be a bounded list.

      The broker also needs enhancement to offer at least two options when a queue builds up:

      1) throttle the publisher. This is likely to be something that is desirable for "public" queues.

      2) disconnect the slow consumer. This will be used when dealing with private queues - typically bound to a topic exchange.

        Activity

        Hide
        Aidan Skinner added a comment -
        Show
        Aidan Skinner added a comment - See http://cwiki.apache.org/confluence/display/qpid/Broker+job+queue+limits for ways to address this.
        Hide
        Andrew Kennedy added a comment -

        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java

        Show
        Andrew Kennedy added a comment - qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java
        Hide
        Andrew Kennedy added a comment -

        Updated due to repository changes

        Show
        Andrew Kennedy added a comment - Updated due to repository changes
        Hide
        Marnie McCormack added a comment -

        Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.

        Show
        Marnie McCormack added a comment - Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.
        Hide
        Robbie Gemmell added a comment -

        Patch applied

        Show
        Robbie Gemmell added a comment - Patch applied
        Hide
        Robbie Gemmell added a comment -

        Resolving as child tasks are all resolved.

        Show
        Robbie Gemmell added a comment - Resolving as child tasks are all resolved.

          People

          • Assignee:
            Martin Ritchie
            Reporter:
            Robert Greig
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development