[HDDS-3080] Configurable container placement policy may not be compatible with pipeline placement policy

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0.0
• Component/s: SCM
• Target Version/s: 1.5.0
• Assignee: Unassigned
• Reporter: Stephen O'Donnell
• Created: 26/Feb/20 11:25
• Updated: 16/Dec/23 23:43

Description

New pipelines are created on hosts in a rack aware way using PipelinePlacementPolicy, which supports only rack aware and random.

However the Container Placement Policy used by ReplicationManager can be configured between the following using the config option ozone.scm.container.placement.impl:

SCMContainerPlacementRackAware
SCMContainerPlacementCapacity
SCMContainerPlacementRandom

If there is a network topology configured via net.topology.table.file.name, then it would be possible to have an incompatible setting for ozone.scm.container.placement.impl.

We need to consider how to ensure the replication manager and Pipeline Provider always use compatible policies. One option may be to merge the rack aware policies as suggested in HDDS-3079, and also allow Capacity and Random to be used in the pipeline provider.