[HDFS-1606] Provide a stronger data guarantee in the write pipeline

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: datanode, hdfs-client, namenode
• Hadoop Flags: Incompatible change, Reviewed

Release Note

Added two configuration properties, dfs.client.block.write.replace-datanode-on-failure.enable and dfs.client.block.write.replace-datanode-on-failure.policy. Added a new feature to replace datanode on failure in DataTransferProtocol. Added getAdditionalDatanode(..) in ClientProtocol.

• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 31/Jan/11 23:40
• Updated: 23/Jun/11 17:41
• Resolved: 12/Apr/11 17:43

Description

In the current design, if there is a datanode/network failure in the write pipeline, DFSClient will try to remove the failed datanode from the pipeline and then continue writing with the remaining datanodes. As a result, the number of datanodes in the pipeline is decreased. Unfortunately, it is possible that DFSClient may incorrectly remove a healthy datanode but leave the failed datanode in the pipeline because failure detection may be inaccurate under erroneous conditions.

We propose to have a new mechanism for adding new datanodes to the pipeline in order to provide a stronger data guarantee.

Issue Links

• is related to: HDFS-1595 DFSClient may incorrectly detect datanode failure
• is related to: HDFS-1599 Umbrella Jira for Improving HBASE support in HDFS
• is related to: HDFS-265 Revisit append
• is related to: HDFS-1785 Cleanup BlockReceiver and DataXceiver
• is related to: HDFS-1789 Refactor frequently used codes from DFSOutputStream, BlockReceiver and DataXceiver
• is related to: HDFS-1817 Split TestFiDataTransferProtocol.java into two files

Sub-Tasks

• Add a method to BlockPlacementPolicy for not removing the chosen nodes: Resolved
• Transfer RBW between datanodes: Closed
• Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK, instead of using RPC: Closed