Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
When stickiness is considered during range assignments, it is possible that in certain cases where co-partitioning is guaranteed we fail.
An example would be:
Consider two topics T1, T2 with 3 partitions each and three members A, B, C.
Let's say the existing assignment (for whatever reason) is:
A -> T1P0 || B -> T1P1, T2P0, T2P1, T2P2 || C -> T1P2
Now we trigger a rebalance with the following subscriptions where all members are subscribed to both topics everything else is the same
A -> T1, T2 || B -> T1, T2 || C -> T1, T2
Since all the topics have an equal number of partitions and all the members are subscribed to the same set of topics we would expect co-partitioning right so would we want the final assignment returned to be
A -> T1P0, T2P0 || B -> T1P1, T2P1 || C -> T1P2, T2P2
SO currently the client side assignor returns the following but it's because they don't assign sticky partitions
C=[topic1-2, topic2-2], B=[topic1-1, topic2-1], A=[topic1-0, topic2-0]Our
Server side assignor returns:
(The partitions in bold are the sticky partitions)
A=MemberAssignment(targetPartitions={topic2=[1], }}{{{}topic1=[0]}), B=MemberAssignment(targetPartitions={topic2=[0], {}topic1=[1]}), C=MemberAssignment(targetPartitions={topic2=[2], {}topic1=[2]
As seen above co-partitioning is expected but not returned.
Attachments
Issue Links
- links to