[HDFS-11366] Clean up old .ckpt files after saveNamespace

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.6.0
• Component/s: hdfs, namenode
• Target Version/s: 2.9.0
• Assignee: Xiao Chen
• Reporter: Xiao Chen
• Created: 24/Jan/17 20:28
• Updated: 24/Jan/17 22:50
• Resolved: 24/Jan/17 22:50

Description

Checkpoints are done in the NN by writing to fsimage.ckpt_TXID files, and rename to fsimage_TXID files upon success.

If a checkpoint fails half way, the fsimage.ckpt_ file will be left on disk. There is no logic to clean it up at all.

After talking with atm, I understand the historical reason for not immediately cleaning up those files, since they maybe useful for disaster recovery.

But feels like cleaning those ckpt files after a successful checkpoint, with a larger TXID threshold is also safe to do.

Issue Links

• duplicates: HDFS-3716 Purger should remove stale fsimage ckpt files