Qpid
  1. Qpid
  2. QPID-3280

Message throughput suffers in a cluster when TTL set

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10
    • Fix Version/s: 0.11
    • Component/s: C++ Broker
    • Labels:
      None

      Description

      When sending a large number of messages with nonzero TTLs to a cluster, overall message throughput drops by around 20-30% compared to messages with TTL 0.

      1. cluster_ttl.diff
        26 kB
        Andy Goldstein

        Activity

        Andy Goldstein created issue -
        Hide
        Andy Goldstein added a comment -

        Potential patch to resolve issue. Added a new "cluster clock" CPG message that the cluster elder fires every so often (default is 10ms, but is configurable). Each node stores and uses the cluster clock when calculating message expiration time or determining if a message has expired. Also moved the QueueCleaner to the cluster timer when the broker is part of a cluster.

        Show
        Andy Goldstein added a comment - Potential patch to resolve issue. Added a new "cluster clock" CPG message that the cluster elder fires every so often (default is 10ms, but is configurable). Each node stores and uses the cluster clock when calculating message expiration time or determining if a message has expired. Also moved the QueueCleaner to the cluster timer when the broker is part of a cluster.
        Andy Goldstein made changes -
        Field Original Value New Value
        Attachment cluster_ttl.diff [ 12480542 ]
        Andy Goldstein made changes -
        Comment [ Attached potential patch to resolve issue. Added a new "cluster clock" CPG message that the cluster elder fires every so often (default is 10ms, but is configurable). Each node stores and uses the cluster clock when calculating message expiration time or determining if a message has expired. Also moved the QueueCleaner to the cluster timer when the broker is part of a cluster. ]
        Hide
        Alan Conway added a comment -

        Rebased patch on trunk, review on reviewboard https://reviews.apache.org/r/791/

        Show
        Alan Conway added a comment - Rebased patch on trunk, review on reviewboard https://reviews.apache.org/r/791/
        Alan Conway made changes -
        Assignee Alan Conway [ aconway ]
        Hide
        Alan Conway added a comment - - edited

        Fixed on trunk #1136170
        Patch provided by Andy Goldstein

        Show
        Alan Conway added a comment - - edited Fixed on trunk #1136170 Patch provided by Andy Goldstein
        Hide
        Alan Conway added a comment -

        Fixed in revision #1136170

        Show
        Alan Conway added a comment - Fixed in revision #1136170
        Alan Conway made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.11 [ 12316272 ]
        Resolution Fixed [ 1 ]
        Hide
        Alan Conway added a comment -

        As part of this fix, failures regarding "cluster timer task" were also fixed

        Show
        Alan Conway added a comment - As part of this fix, failures regarding "cluster timer task" were also fixed
        Justin Ross made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Alan Conway
            Reporter:
            Andy Goldstein
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development