[#HADOOP-1593] FsShell should work with paths in non-default FileSystem - ASF JIRA

Issue Details (XML | Word | Printable)

Key:	HADOOP-1593
Type:	New Feature
Status:	Closed
Resolution:	Fixed
Priority:	Major
Assignee:	Mahadev konar
Reporter:	Doug Cutting
Votes:	0
Watchers:	0

Operations

If you were logged in you would be able to see more operations.

Hadoop Common

FsShell should work with paths in non-default FileSystem

Created: 11/Jul/07 05:58 PM Updated: 21/May/08 08:05 PM

Return to search

Component/s:

fs

Affects Version/s:

None

Fix Version/s:

0.17.0

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	patch_1593.patch	2008-02-09 04:45 AM	Mahadev konar	28 kB
	patch_1593_1.patch	2008-02-11 08:09 PM	Mahadev konar	27 kB

Release Note:

« Hide

This bug allows non default path to specifeid in fsshell commands.

So, you can now specify hadoop dfs -ls hdfs://remotehost1:port/path
and hadoop dfs -ls hdfs://remotehost2:port/path without changing the config.

Resolution Date:

11/Feb/08 10:33 PM

Description

If the default filesystem is, e.g., hdfs://foo:8888/, one should still be able to do 'bin/hadoop fs -ls hdfs://bar:9999/' or 'bin/hadoop fs -ls s3://cutting/foo'. Currently these generate a filesystem mismatch exception. This is because FsShell assumes that all paths are in the default FileSystem. Rather, the default filesystem should only be used for paths that do not specify a FileSystem. This would easily be accomplished by using Path#getFileSystem().

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Mahadev konar added a comment - 09/Feb/08 04:45 AM

this patch fixes the issue and has unit tests to test case of FSShell being used by non default pathnames.
So,

hadoop dfs -lsr hdfs://host:port/path

works on a non default filesystem
and the same is true for all the other fs shell commands.

[ Permlink | « Hide ]

dhruba borthakur added a comment - 11/Feb/08 07:53 PM

+1 Code looks good. There are a few blank-line changes in this patch that can be safely removed.

[ Permlink | « Hide ]

Mahadev konar added a comment - 11/Feb/08 08:09 PM

removing unwanted spaces in this patch.

[ Permlink | « Hide ]

Hadoop QA added a comment - 11/Feb/08 10:04 PM

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12375266/patch_1593_1.patch
against trunk revision 619744.

@author +1. The patch does not contain any @author tags.

tests included +1. The patch appears to include 3 new or modified tests.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1775/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1775/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1775/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1775/console

This message is automatically generated.

[ Permlink | « Hide ]

dhruba borthakur added a comment - 11/Feb/08 10:33 PM

I just committed this. Thanks Mahadev!

[ Permlink | « Hide ]

Hudson added a comment - 12/Feb/08 12:10 PM

Integrated in Hadoop-trunk #398 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/398/)