Minor Description
The tests for the IncrementalCooperativeAssignor class provide a moderate level of coverage and cover some non-trivial cases, but there are some areas for improvement that will allow us to iterate on the assignment logic for Kafka Connect faster and with greater confidence.
These improvements include:
- Adding reusable utility methods to assert that a cluster's assignment is balanced (the difference in the number of connectors and tasks assigned to any two workers is at most one) and complete (all connectors and tasks are assigned to a worker)
- Removing the existing assertAssignment methods and replacing them with a more fine-grained alternative that allows for more granular assertions about the number of tasks/connectors assigned/revoked from each worker during a round of rebalance, instead of the total for the entire cluster
- Adding a reusable utility method to assert the current distribution of connectors and tasks across the cluster
- Decomposing large portions of repeated code for simulating a round of rebalancing into a reusable utility method
- Renaming variable names to improve accuracy/readability (the expectedMemberConfigs field, for example, is pretty poorly named)
But other improvements may be added in a pull request that addresses the above as they come up.
Attachments
Issue Links
- relates to
-
KAFKA-12495 Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor
-
- Resolved
-
- links to