[HADOOP-5145] Balancer sometimes runs out of memory after days or weeks running

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.20.0
• Hadoop Flags: Reviewed
• Assignee: Hairong Kuang
• Reporter: Hairong Kuang
• Created: 30/Jan/09 01:22
• Updated: 08/Jul/09 16:43
• Resolved: 05/Mar/09 18:32

Description

The culprit is a HashMap called MovedBlocks. By design this map does not get cleaned up between iterations. This is because the deletion of source replicas is done by NN. When next iteration starts, source replicas may not have been deleted, Balancer does not want to schedule them to move again. To prevent running out of memory, Balancer should expire/clean the movedBlocks from some iterations back.