[HDFS-159] Block reports should be processed offline

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Sameer Paranjpye
• Reporter: Owen O'Malley
• Created: 06/Oct/06 21:07
• Updated: 18/Jan/12 09:36
• Resolved: 18/Jan/12 09:36

Description

The namenode's handling of block reports should be done from a batch queue instead of while the rpc call from the client is waiting. I'd propose:

void blockReport(DataNodeRegistration, Block[]) throws IOException;

instead of passing back obsolete blocks they can be handed back on later heartbeat.

The blockReport calls should just queue the registration and list of blocks and there should be a thread that processes the block reports when possible.

The goal is to prevent rpc timeouts and therefore retransmissions of the block lists.

Issue Links

• duplicates: HDFS-2379 0.20: Allow block reports to proceed without holding FSDataset lock