Affects Version/s: 0.20.2, 0.21.0
I was trying to discover details about the Secondary NameNode, and came across the description below in the HDFS design doc.
The NameNode keeps an image of the entire file system namespace and file Blockmap in memory. This key metadata item is designed to be compact, such that a NameNode with 4 GB of RAM is plenty to support a huge number of files and directories. When the NameNode starts up, it reads the FsImage and EditLog from disk, applies all the transactions from the EditLog to the in-memory representation of the FsImage, and flushes out this new version into a new FsImage on disk. It can then truncate the old EditLog because its transactions have been applied to the persistent FsImage. This process is called a checkpoint. In the current implementation, a checkpoint only occurs when the NameNode starts up. Work is in progress to support periodic checkpointing in the near future.
Note that this directly conflicts with information in the hdfs user guide, http://hadoop.apache.org/common/docs/r0.20.2/hdfs_user_guide.html#Secondary+NameNode
I haven't done a thorough audit of that doc-- I only noticed the above inaccuracy.
|Status||Patch Available [ 10002 ]||Resolved [ 5 ]|
|Fix Version/s||0.21.1 [ 12315271 ]|
|Fix Version/s||0.22.0 [ 12314241 ]|
|Fix Version/s||0.23.0 [ 12315571 ]|
|Resolution||Fixed [ 1 ]|
|Assignee||Joe Crobak [ joecrobak ]|
|Status||Open [ 1 ]||Patch Available [ 10002 ]|