Details
-
Task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Current, CRUSH (also multi-round CRUSH) strategy provides even partition distribution based on statistically randomization. All CRUSH based strategy works well when the cluster is reasonably large.
However, in some of our small production clusters, we find the distribution is not good as expected. The difference between heavy load node and idle node could be 30% in our case. Given we assign resource according to the max load, we are wasting 30% of resource on the idle nodes. And it could be worse in other clusters.
We need to find a new algorithm or improve CRUSH for better evenness for small clusters.
Note that, meanwhile, we should keep the good features such as minimum movements, deterministic calculating result.