Description
HDFS replicas placement strategy guarantees that the replicas of a block exist on at least two racks when its replication factor is greater than one. But fsck still reports that the replicas of some blocks end up on one rack.
The cause of the problem is that decommission and corruption handling only check the block's replication factor but not the rack requirement. When an over-replicated block loses a replica due to decomission, corruption, or heartbeat lost, namenode does not take any action to guarantee that remaining replicas are on different racks.
Attachments
Attachments
Issue Links
- is related to
-
HDFS-37 An invalidated block should be removed from the blockMap
- Resolved
-
HDFS-1480 All replicas of a block can end up on the same rack when some datanodes are decommissioning.
- Resolved
-
HDFS-3256 HDFS considers blocks under-replicated if topology script is configured with only 1 rack
- Closed
-
HDFS-1562 Add rack policy tests
- Closed
- relates to
-
HADOOP-4489 Blocks have been scheduled to delete by NameNode but are not deleted on the DataNode
- Closed