[HDFS-1108] Log newly allocated blocks

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: HA branch (HDFS-1623), 1.1.1
• Component/s: ha, namenode
• Target Version/s: 1.1.1
• Assignee: Todd Lipcon
• Reporter: Dhruba Borthakur
• Created: 23/Apr/10 23:30
• Updated: 03/Dec/12 07:33
• Resolved: 24/Aug/12 00:28

Description

The current HDFS design says that newly allocated blocks for a file are not persisted in the NN transaction log when the block is allocated. Instead, a hflush() or a close() on the file persists the blocks into the transaction log. It would be nice if we can immediately persist newly allocated blocks (as soon as they are allocated) for specific files.

Issue Links

• blocks: HBASE-2108 [HA] hbase cluster should be able to ride over hdfs 'safe mode' flip and namenode restart/move
• blocks: HDFS-2064 Warm HA NameNode going Hot
• is duplicated by: HDFS-978 Record every new block allocation of a file into the transaction log.
• is related to: HDFS-1975 HA: Support for sharing the namenode state from active to standby.
• is superceded by: HDFS-2602 NN should log newly-allocated blocks without losing BlockInfo
• relates to: HDFS-2602 NN should log newly-allocated blocks without losing BlockInfo
• relates to: HDFS-3630 Modify TestPersistBlocks to use both flush and hflush