Uploaded image for project: 'Qpid'
  1. Qpid
  2. QPID-1447

Broker does not handle with slow consumers effectively

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: M3
    • Fix Version/s: 0.7
    • Component/s: Java Broker
    • Labels:
      None
    • Environment:

      Any

      Description

      Please see this qpid-user thread for further context: http://markmail.org/message/vgtssb2ivyttnbsf

      The qpid java broker does not deal effectively with slow consumers. This is a combination of two issues: a MINA issue and a lack of support in qpid itself.

      The MINA issue is similar to the "fast producer" issue. In this case, the data is stored in the MINA session queues, and does not appear in the queue counts which makes it very difficult for users to track down. Each MINA session event queue needs to be a bounded list.

      The broker also needs enhancement to offer at least two options when a queue builds up:

      1) throttle the publisher. This is likely to be something that is desirable for "public" queues.

      2) disconnect the slow consumer. This will be used when dealing with private queues - typically bound to a topic exchange.

        Activity

        Hide
        aidan Aidan Skinner added a comment -
        Show
        aidan Aidan Skinner added a comment - See http://cwiki.apache.org/confluence/display/qpid/Broker+job+queue+limits for ways to address this.
        Hide
        andrew.kennedy Andrew Kennedy added a comment -

        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java
        qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java

        Show
        andrew.kennedy Andrew Kennedy added a comment - qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionPolicyConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/configuration/plugin/SlowConsumerDetectionQueueConfiguration.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/SlowConsumerDetection.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/server/virtualhost/plugin/policies/TopicDeletePolicy.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPlugin.java qpid/java/broker-plugins/experimental/slowconsumerdetection/src/main/java/org/apache/qpid/slowconsumerdetection/policies/SlowConsumerPolicyPluginFactory.java
        Hide
        andrew.kennedy Andrew Kennedy added a comment -

        Updated due to repository changes

        Show
        andrew.kennedy Andrew Kennedy added a comment - Updated due to repository changes
        Hide
        marnie Marnie McCormack added a comment -

        Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.

        Show
        marnie Marnie McCormack added a comment - Work in progress is for the slow consumer disconnect element of this JIRA, since the Java broker now has support for Producer Side Flow Control.
        Hide
        gemmellr Robbie Gemmell added a comment -

        Patch applied

        Show
        gemmellr Robbie Gemmell added a comment - Patch applied
        Hide
        gemmellr Robbie Gemmell added a comment -

        Resolving as child tasks are all resolved.

        Show
        gemmellr Robbie Gemmell added a comment - Resolving as child tasks are all resolved.

          People

          • Assignee:
            ritchiem Martin Ritchie
            Reporter:
            rgreig Robert Greig
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development