[HDFS-14560] Allow block replication parameters to be refreshable

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: namenode
• Target Version/s: 3.3.0
• Hadoop Flags: Reviewed
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 11/Jun/19 10:50
• Updated: 16/Jul/22 09:25
• Resolved: 14/Jun/19 01:29

Description

There are 3 key parameters that control the speed of block replication across the cluster:

dfs.namenode.replication.max-streams
dfs.namenode.replication.max-streams-hard-limit
dfs.namenode.replication.work.multiplier.per.iteration

These are used when decomissioning nodes and when under replicated blocks are being recovered across the cluster. There are times when it may be desirable to increase the speed of replication and then reduce it again (eg during off peak hours) without restarting the namenode.

Therefore it would be good to be able to reconfigure / refresh these parameters dynamically without a namenode restart.

This Jira is to allow these parameters to be refreshed at runtime without a restart.

Issue Links

• is duplicated by: HDFS-15244 Make Decommission related configurations runtime refreshable
• is related to: HDFS-16663 EC: Allow block reconstruction pending timeout refreshable to increase decommission performance