-
Type:
Improvement
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.20-append
-
Fix Version/s: 0.20-append, 0.20.205.0, 0.21.0
-
Component/s: hdfs-client, namenode
-
Labels:None
-
Hadoop Flags:Incompatible change, Reviewed
created from hdfs-200.
If during a write, the dfsclient sees that a block replica location for a newly allocated block is not-connectable, it re-requests the NN to get a fresh set of replica locations of the block. It tries this dfs.client.block.write.retries times (default 3), sleeping 6 seconds between each retry ( see DFSClient.nextBlockOutputStream).
This setting works well when you have a reasonable size cluster; if u have few datanodes in the cluster, every retry maybe pick the dead-datanode and the above logic bails out.
Our solution: when getting block location from namenode, we give nn the excluded datanodes. The list of dead datanodes is only for one block allocation.
- blocks
-
HBASE-1876 DroppedSnapshotException when flushing memstore after a datanode dies
-
- Closed
-
- is related to
-
HDFS-200 In HDFS, sync() not yet guarantees data available to the new readers
-
- Closed
-