[HBASE-15531] Favored Nodes Enhancements

• Type: Umbrella
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Assignee: Unassigned
• Reporter: Francis Christopher Liu
• Created: 25/Mar/16 17:30
• Updated: 23/Jun/22 19:11
• Resolved: 23/Jun/22 19:10

Description

We been working on enhancing favored nodes at Yahoo! See draft document. Feel free to comment. I'll add more info.

https://docs.google.com/document/d/1948RKX_-kGUNOHjiYFiKPZnmKWybJkahJYsNibVhcCk/edit?usp=sharing

These enhancements have recently started running in production.

Issue Links

• is duplicated by: HBASE-21014 Improve Stochastic Balancer to write HDFS favoured node hints for region primary blocks to avoid destroying data locality if needing to use HDFS Balancer
• relates to: HBASE-17760 HDFS Balancer doc is misleading
• relates to: HBASE-13021 Pin regions blocks to avoid HDFS balancing
• supercedes: HBASE-4755 HBase based block placement in DFS
• links to: Favored Nodes v2 - design document

Sub-Tasks

• Support first assignment of split daughters to non-parent RS: Closed
• core favored nodes enhancements: Closed
• Add RSGroup Favored Balancer: Closed
• FavoredNodes - Split/Merge code paths: Closed
• Add FavoredStochasticLoadBalancer and FN Candidate generators: Closed
• FN Interfaces for writing tools for monitoring/operations: Closed
• Refactor FavoredNodePlan to use regionNames as keys: Closed
• Implement Chore to sync FN info from Master to RegionServers: Closed
• FavoredNodes should not apply to system tables: Closed
• FN info should be cleaned up on region/table cleanup: Closed
• Add FavoredNode checks to hbck -checkFavoredNodes: Closed
• Split/Merge - Update of FN info along with regionState change: Closed
• FN updates during region merge (follow up to Procedure v2 merge): Closed
• Refactor StartcodeAgnosticServerName so it doesn't extend ServerName: Closed
• FN should use datanode port from hdfs configuration: Closed
• Add integration tests for FavoredNodes feature: Closed
• Adding FN documentation to reference guide: Closed
• Move table to another group (add -migrateTertiary): Closed
• Refactor HRegionServer to use regionNames as keys for FN: Closed
• Admin API to update favored nodes in Master: Closed
• Tools/API to read favored nodes for region(s): Closed
• Tools/Admin API to dump the replica load of server(s): Closed
• Follow up enhancements for Favored nodes pickers: Closed
• Admin API to removeFavoredNode on node decommission: Closed
• Admin API to check for favored nodes: Closed
• Admin API to redistribute Favored Nodes: Closed
• Admin API to do complete redistribute of favored nodes: Closed