[HDDS-9889] Refactor tests related to dynamical adaptation for datanode limits in ContainerBalancer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.3.0, 1.4.0
• Fix Version/s: 2.0.0, 1.4.1
• Component/s: Balancer, SCM, test
• Labels: pull-request-available

Environment

OS: Ubuntu 22.04.3 LTS,  Arch: 64-bit, Processor: Intel Core i7-9750H CPU 2.6GHz x 12

• Target Version/s: 2.0.0, 1.4.1
• Assignee: Andrei Mikhalev
• Reporter: Andrei Mikhalev
• Created: 11/Dec/23 07:51
• Updated: 15/Oct/24 17:47
• Resolved: 01/May/24 12:08

Description

Dynamical adaptation (introduced in HDDS-5526) for datanodes.involved.max.percentage.per.iteration in container balancer doesn't work well in some cases.

Sometimes the number of under-utilized nodes may not be sufficient to satisfy the limit about the max percent of datanodes participating in the balance iteration (datanodes.involved.max.percentage.per.iteration).

Thus, collections of source and target datanodes are reset and balancing is skipped.

The issue it can be easily detected when cluster has few nodes (< 10), for example 4 or 5.

 

Solution: 
 siddhant wrote a small documentaion with some facing details about container balancer.

 

Issue Links

• blocks: HDDS-10699 Refactor ContainerBalancerTask and tests in TestContainerBalancerTask
• links to: GitHub Pull Request #5758