[HBASE-21014] Improve Stochastic Balancer to write HDFS favoured node hints for region primary blocks to avoid destroying data locality if needing to use HDFS Balancer

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.2
• Component/s: Balancer
• Labels: balancer
• Assignee: Unassigned
• Reporter: Hari Sekhon
• Created: 06/Aug/18 11:00
• Updated: 22/Apr/19 21:01

Description

Improve Stochastic Balancer to include the HDFS region location hints to avoid HDFS Balancer destroying data locality.

Right now according to a mix of docs, jiras and mailing list info it appears that one must change

hbase.master.loadbalancer.class

to the org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer as it looks like this functionality is only within FavoredNodeBalancer and not the standard Stochastic Balancer.

http://hbase.apache.org/book.html#_hbase_and_hdfs

This is not ideal because we'd still like to use all the heuristics and work that has gone in the Stochastic Balancer which I believe right now is the best and most mature HBase balancer.

See also the linked Jiras and this discussion:

http://apache-hbase.679495.n3.nabble.com/HDFS-Balancer-td4086607.html

Issue Links

• duplicates: HBASE-15531 Favored Nodes Enhancements
• relates to: HBASE-7932 Do the necessary plumbing for the region locations in META table and send the info to the RegionServers
• relates to: HBASE-7942 Make use of the plumbing in HBASE-7932 to provide location hints when region files are created
• relates to: HBASE-17760 HDFS Balancer doc is misleading
• relates to: HDFS-6133 Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover.