[HDFS-6133] Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: balancer & mover, datanode
• Hadoop Flags: Reviewed
• Release Note: Add a feature for replica pinning so that when a replica is pinned in a datanode, it will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by "dfs.datanode.block-pinning.enabled", where the default is false.
• Assignee: yunjiong zhao
• Reporter: yunjiong zhao
• Created: 20/Mar/14 22:38
• Updated: 09/Aug/18 09:10
• Resolved: 11/Feb/15 23:18

Description

Currently, run Balancer will destroying Regionserver's data locality.
If getBlocks could exclude blocks belongs to files which have specific path prefix, like "/hbase", then we can run Balancer without destroying Regionserver's data locality.

Issue Links

• breaks: HDFS-7954 TestBalancer#testBalancerWithPinnedBlocks failed on Windows
• is related to: HBASE-21014 Improve Stochastic Balancer to write HDFS favoured node hints for region primary blocks to avoid destroying data locality if needing to use HDFS Balancer
• is related to: HDFS-4420 Provide a way to exclude subtree from balancing process
• is related to: HDFS-11401 Reduce verbosity of logs with favored nodes and block pinning enabled
• relates to: HBASE-4755 HBase based block placement in DFS
• relates to: HDFS-9889 Update balancer/mover document about HDFS-6133 feature
• relates to: HDFS-7849 Update documentation for enabling a new feature in rolling upgrade
• relates to: HDFS-7759 Provide existence-of-a-second-file implementation for pinning blocks on Datanode