[HADOOP-64] DataNode should be capable of managing multiple volumes

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.2.0
• Fix Version/s: 0.6.0
• Assignee: Milind Barve
• Reporter: Sameer Paranjpye
• Created: 07/Mar/06 07:05
• Updated: 08/Jul/09 16:41
• Resolved: 05/Sep/06 22:14

Description

The dfs Datanode can only store data on a single filesystem volume. When a node runs its disks JBOD this means running a Datanode per disk on the machine. While the scheme works reasonably well on small clusters, on larger installations (several 100 nodes) it implies a very large number of Datanodes with associated management overhead in the Namenode.

The Datanod should be enhanced to be able to handle multiple volumes on a single machine.

Issue Links

• duplicates: HADOOP-74 hash blocks into dfs.data.dirs
• duplicates: HADOOP-257 starting one data node thread to manage multiple data directories
• incorporates: HADOOP-50 dfs datanode should store blocks in multiple directories