[#HADOOP-4745] EC2 scripts should configure Hadoop to use all available disks on large instances

Hadoop Common

EC2 scripts should configure Hadoop to use all available disks on large instances

Created: 01/Dec/08 02:50 PM Updated: 15/Dec/08 07:51 PM

Return to search

Component/s:

contrib/ec2

Affects Version/s:

0.19.0

Fix Version/s:

None

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	hadoop-4745.patch	2008-12-01 02:55 PM	Tom White	4 kB

Issue Links:

Reference

This issue is related to:

HADOOP-4382 Run Hadoop sort benchmark on Amazon EC2

Description

« Hide

The Hadoop configuration on EC2 currently always uses a single disk, even when more are available (http://docs.amazonwebservices.com/AWSEC2/2007-08-29/DeveloperGuide/instance-storage.html). Performance is significantly boosted by using all the available disks, so we should configure Hadoop to use them automatically.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Tom White added a comment - 01/Dec/08 02:55 PM

Patch that changes the instance boot script to: query for the instance type, format and mount available disks, configure hadoop-site.xml appropriately according to instance type.