[HADOOP-2549] hdfs does not honor dfs.du.reserved setting

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.14.4
• Fix Version/s: 0.16.0

Environment

FC Linux.

• Assignee: Hairong Kuang
• Reporter: Joydeep Sen Sarma
• Created: 08/Jan/08 17:06
• Updated: 06/Jan/11 09:07
• Resolved: 22/Jan/08 20:03

Description

running 0.14.4. one of our drives is smaller and is always getting disk full. i reset the disk reservation to 1Gig - but it was filled quickly again.

i put in some tracing in getnextvolume. the blocksize argument is 0. so every volume (regardless of available space) qualifies. here's the trace:

/* root disk chosen with 0 available bytes. format is <available>:<blocksize>*/
2008-01-08 08:08:51,918 WARN org.apache.hadoop.dfs.DataNode: Volume /var/hadoop/tmp/dfs/data/current:0:0

/* some other disk chosen with 300G space. */
2008-01-08 08:09:21,974 WARN org.apache.hadoop.dfs.DataNode: Volume /mnt/d1/hdfs/current:304725631026:0

i am going to default blocksize to something reasonable when it's zero for now.

this is driving us nuts since our automounter starts failing when we run out of space. so everything's broke.

Issue Links

• relates to: HDFS-1564 Make dfs.datanode.du.reserved configurable per volume