Uploaded image for project: 'Apache RocketMQ'
  1. Apache RocketMQ
  2. ROCKETMQ-67

Consistent Hash allocate strategy support

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0-incubating
    • Component/s: rocketmq-client
    • Labels:
      None

      Description

      For now, the average allocate strategy is very sensitive when clients register and unrigister.

      A Consistent Hash allocate strategy option is valueable for the developers who care more about latency stabilization and messages duplication.

      Intentions:
      The default AllocateMessageQueueStrategy is averaging strategy which allocate queue to consumer as evenly as possible. Whenever queues numbers or consumer numbers changed, say a new consumer starts or an old consumer shutdowns, a rehashing will be triggered then almost all consumer suffered from this that they will rebalance to drop old queues and get new queues.

      And that will cause

      message latency from producer to consumer increases at the moment when consumer/queue numbers change, even when they scale up.

      messages will be duplicated significantly since the offset may not be persisted to broker and that queue is assigned to another consumer to pull messages from.

      This is especially significant when they have tens of consumer instances and scale-up or deployment is often.

      Consistent Hash strategy to allocate queue is a good choice for these users.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Jaskey Jaskey Lam
                Reporter:
                Jaskey Jaskey Lam
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: