[#MAHOUT-118] Mahout needs to respect the file system type when getting a FileSystem for an input or output path

Mahout

Mahout needs to respect the file system type when getting a FileSystem for an input or output path

Created: 16/Apr/09 08:22 PM Updated: 18/Nov/09 02:05 PM

Return to search

Component/s:

Classification, Clustering, Collaborative Filtering, Genetic Algorithms, Matrix

Affects Version/s:

0.1, 0.2

Fix Version/s:

0.1, 0.2

Time Tracking:

Original Estimate:

24h

Remaining Estimate:

24h

Time Spent:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	getfs.patch	2009-04-16 08:51 PM	Stephen Green	33 kB

Environment:

Mac OS X 10.5 and Amazon's Elastic MapReduce

Resolution Date:

19/Apr/09 11:18 PM

Description

« Hide

All of the uses of org.apache.hadoop.fs.FileSystem.get use the single argument version that takes a job configuration. This will always return the default file system type (which is usually HDFS), rather than using the file system type used in the URIs for the input or output paths. This is particularly a problem on Amazon's Elastic MapReduce where the input and output data typically reside in a org.apache.hadoop.fs.s3native.NativeS3FileSystem.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Stephen Green made changes - 16/Apr/09 08:51 PM

Field	Original Value	New Value
Attachment		getfs.patch [ 12405697 ]

Jeff Eastman made changes - 16/Apr/09 09:10 PM

Assignee

Jeff Eastman [ jeastman ]

Jeff Eastman made changes - 19/Apr/09 11:18 PM

Resolution		Fixed [ 1 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Grant Ingersoll made changes - 18/Nov/09 02:05 PM

Status

Resolved [ 5 ]

Closed [ 6 ]