[HDFS-9608] Disk IO imbalance in HDFS with heterogeneous storages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Target Version/s: 2.9.0
• Assignee: Wei Zhou
• Reporter: Wei Zhou
• Created: 04/Jan/16 02:06
• Updated: 15/Feb/18 06:20
• Resolved: 17/Feb/16 19:30

Description

Currently RoundRobinVolumeChoosingPolicy use a shared index to choose volumes in HDFS with heterogeneous storages, this leads to non-RR choosing mode for certain type of storage.
Besides, it uses a shared lock for synchronization which limits the concurrency of volume choosing process. Volume choosing threads that operating on different storage types should be able to run concurrently.