Qpid
  1. Qpid
  2. QPID-3780

[Java Broker] Using multiple consumers on a large (e.g. 1million entries+) queue may become excessively slow

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14
    • Fix Version/s: 0.15
    • Component/s: Java Broker
    • Labels:
      None

      Description

      If there are multiple consumers on a queue, a certain percentage of acknowledgements will not be at the head of the queue because consumers will not act in lock step. Every time a message not at the head of the queue is consumed we increment the scavenge count to indicate that we should scan the queue for deleted entries and remove them. If the queue is very large then this will take a long time. Clearing down a very large queue with multiple consumers will thus become very expensive.

      We can reduce the cost of scavenging by keeping a record of the high watermark of deleted entries that weren't at the head of the queue, and only scavenging the queue to a point beyond the high watermark

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        1m 13s 1 Rob Godfrey 25/Jan/12 14:40
        In Progress In Progress Reviewable Reviewable
        5s 1 Rob Godfrey 25/Jan/12 14:40
        Reviewable Reviewable Resolved Resolved
        19h 1m 1 Robbie Gemmell 26/Jan/12 09:42
        Resolved Resolved Closed Closed
        1112d 10h 24m 1 Rob Godfrey 11/Feb/15 20:06
        Rob Godfrey made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Robbie Gemmell made changes -
        Status Ready To Review [ 10006 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Robbie Gemmell added a comment -

        Changes look good to me.

        Show
        Robbie Gemmell added a comment - Changes look good to me.
        Rob Godfrey made changes -
        Status In Progress [ 3 ] Ready To Review [ 10006 ]
        Rob Godfrey made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Rob Godfrey created issue -

          People

          • Assignee:
            Rob Godfrey
            Reporter:
            Rob Godfrey
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1h
              1h
              Remaining:
              Remaining Estimate - 1h
              1h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development