[HDFS-2422] The NN should tolerate the same number of low-resource volumes as failed volumes

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: namenode
• Target Version/s: 0.23.0
• Hadoop Flags: Reviewed
• Assignee: Aaron Myers
• Reporter: Jeff Bean
• Created: 08/Oct/11 22:29
• Updated: 15/Nov/11 00:53
• Resolved: 11/Oct/11 00:38

Description

We encountered a situation where the namenode dropped into safe mode after a temporary outage of an NFS mount.

At 12:10 the NFS server goes offline

Oct 8 12:10:05 <namenode> kernel: nfs: server <nfs host> not responding, timed out

This caused the namenode to conclude resource issues:

2011-10-08 12:10:34,848 WARN org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space available on volume '<nfs host>' is 0, which is below the configured reserved amount 104857600

Temporary loss of NFS mount shouldn't cause safemode.

Issue Links

• blocks: HDFS-1594 When the disk becomes full Namenode is getting shutdown and not able to recover