[HDFS-5134] NameNode improvements to per-storage block processing

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Heterogeneous Storage (HDFS-2832)
• Fix Version/s: Heterogeneous Storage (HDFS-2832)
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Arpit Agarwal
• Created: 27/Aug/13 18:13
• Updated: 18/Oct/13 23:08
• Resolved: 08/Sep/13 22:54

Description

Few tasks out of the NameNode change for multiple storage (HDFS-4987):

1. blockContentsStale should be per storage.
2. DatanodeDescriptor#firstBlockReport should also be per storage.
3. BlockManager#processReport can call processFirstBlockReport per storage now.
4. Synchronization for DatanodeDescriptor#numBlocks (mentioned by Nicholas in 4987).

Issue Links

• depends upon: HDFS-4987 Namenode changes to track multiple storages
• relates to: HDFS-5377 Heartbeats from Datandode should include one storage report per storage directory
• relates to: HDFS-5390 Send one incremental block report per storage directory