[HDFS-4989] Balancer needs to consider storage type in balancing

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: balancer & mover, datanode, namenode
• Assignee: Junping Du
• Reporter: Suresh Srinivas
• Created: 12/Jul/13 22:33
• Updated: 25/Oct/19 20:27
• Resolved: 21/Jan/15 06:15

Description

Balancer needs to balance with in a storage tier. A couple of works:

• add an option of storageType in balancer CLI to balance data on a specific storage type.
• add storageType to BalancingPolicy
• need new API (getDatanodeReport(StorageType) peer with getDatanodeReport() before) in ClientProtocol to get storage related info from NN so that balancer can choose storages as <source, target> balancing pair.
• update chooseDatanodes() algorithm to choose storages on the same DN first in setting up <under-utilization, over-load> pairs.

Issue Links

• depends upon: HDFS-4990 Block placement for storage types
• depends upon: HDFS-5178 DataTransferProtocol changes for supporting mutiple storages per DN
• duplicates: HDFS-6685 Balancer should preserve storage type of replicas