[HDFS-10214] Checkpoint Can not be done by StandbyNameNode.Because checkpoint may cause DataNode blockReport.blockReceivedAndDeleted.heartbeat rpc timeout when the object num > 100000000.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.5.0, 2.6.4
• Component/s: ha, namenode

Environment

500 DataNode.

137407265 files and directories, 129614074 blocks = 267021339 total filesystem object(s)

• Assignee: Unassigned
• Reporter: ChenFolin
• Due: 24/Mar/16
• Created: 25/Mar/16 05:15
• Updated: 05/Jan/17 21:01
• Resolved: 27/Apr/16 06:35

Description

The current Cluster status :
137407265 files and directories, 129614074 blocks = 267021339 total filesystem object(s).

The checkpoint save namespace cost more than 5 min.

DataNode rpc timeout.

Standby NameNode skip the DataNode rpc request(because datanode rpc timeout , datanode close the socket channel).

There are many corrupt files when failover.

So, Checkpoint may be done by other component, not Standby NameNode.

Issue Links

• is duplicated by: HDFS-7097 Allow block reports to be processed during checkpointing on standby name node