Issue Details (XML | Word | Printable)

Key: HADOOP-2194
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Mahadev konar
Reporter: Mukund Madhugiri
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

dfs cat on a file that does not exist throws a java IOException

Created: 12/Nov/07 07:13 PM   Updated: 08/Jul/09 04:42 PM
Return to search
Component/s: None
Affects Version/s: 0.15.1, 0.16.0
Fix Version/s: 0.17.0

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works patch_2194_1.patch 2008-01-31 09:28 PM Mahadev konar 0.5 kB
Text File Licensed for inclusion in ASF works patch_2194_2.patch 2008-02-05 07:53 PM Mahadev konar 1 kB
Text File Licensed for inclusion in ASF works patch_2194_3.patch 2008-02-05 10:56 PM Mahadev konar 1 kB
Text File Licensed for inclusion in ASF works patch_2194_4.patch 2008-02-05 11:13 PM Mahadev konar 2 kB
Text File Licensed for inclusion in ASF works patch_2194_5.patch 2008-02-06 07:37 PM Mahadev konar 4 kB

Resolution Date: 07/Feb/08 02:48 PM


 Description  « Hide
The dfs cat command throws an IOException when the file does not exist:

bin/hadoop dfs -cat doesnotexist
cat: java.io.IOException: Cannot open filename<home directory>/doesnotexist]

In Linux, if a file does not exist, cat displays this:
cat: doesnotexist: No such file or directory



 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Mahadev konar added a comment - 31/Jan/08 09:28 PM
here is a patch that fixes the issue.

Hadoop QA added a comment - 02/Feb/08 03:56 AM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12374503/patch_2194_1.patch
against trunk revision 616796.

@author +1. The patch does not contain any @author tags.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1726/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1726/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1726/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1726/console

This message is automatically generated.


Hairong Kuang added a comment - 04/Feb/08 10:30 PM
This patch adds an additional RPC to the namenode. I think it would be better if DFS.open can throw a FileNotFound exception when the file does not exist.

Mahadev konar added a comment - 04/Feb/08 10:37 PM
an additional RPC is added to FSShell which is not programtically used by the users. so there would not be a performance hit for the api. open throwing an exception would inlvolve parsing the remote exception string which I think is not a good idea.

Hairong Kuang added a comment - 05/Feb/08 04:48 AM
Shell commands can be used in the batch mode. Some users programtically use FsShell to take advantage of the features like trash bin.

Mahadev konar added a comment - 05/Feb/08 07:53 PM
this is a patch with doing a rcp call and parses the remoteexception.

Mahadev konar added a comment - 05/Feb/08 07:53 PM
i meant without doing a rpc call

Hadoop QA added a comment - 05/Feb/08 08:58 PM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12374811/patch_2194_2.patch
against trunk revision 616796.

@author +1. The patch does not contain any @author tags.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1744/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1744/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1744/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1744/console

This message is automatically generated.


Raghu Angadi added a comment - 05/Feb/08 10:53 PM
This is already doing a fs.isDirectory(), so we can find if the file exists without an extra rpc call and without parsing IOException. We need to use fs.getFileStatus() instead of fs.isDirectory().

Mahadev konar added a comment - 05/Feb/08 10:56 PM
patch that reduces a few lines and keeps track of the remote exception in filenotfound exception.

Mahadev konar added a comment - 05/Feb/08 11:13 PM
patch that uses getFIleStatus and fixes filenotfoundexception thrown by getFIleStatus.

Tsz Wo (Nicholas), SZE added a comment - 05/Feb/08 11:47 PM
patch_2194_4.patch: +1

Hairong Kuang added a comment - 06/Feb/08 12:29 AM
+1 The code looks good.

One more thought... A fundamental fix to this problem should let FileSystem.open throws a FileNotFoundException when the input path is a directory or the path does not exist. Then there is no need to check if the path is a directory before calling open.


Hadoop QA added a comment - 06/Feb/08 12:59 AM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12374832/patch_2194_4.patch
against trunk revision 616796.

@author +1. The patch does not contain any @author tags.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1745/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1745/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1745/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1745/console

This message is automatically generated.


Nigel Daley added a comment - 06/Feb/08 03:34 AM
-1. No regression test.

dhruba borthakur added a comment - 06/Feb/08 03:25 PM
This is a generic issue, what I mean is that there are many portions of client code that needs to map a RemoteException to a more specifiex Exception. Most client side code somehow maps a Remote Exception to a printable string that the user can understand. Maybe it makes more sense to have a new method in the RemoteException class itself that parses the remote exception string and generates an appropriate exception (and/or message) that is relevant to the Client code. This will ensure that the mapping of the RemoteException to the client exception/message remain localized to one place instead of being scattered all around in the client code.

Mahadev konar added a comment - 06/Feb/08 07:37 PM
last patch with a unit test .

Hadoop QA added a comment - 06/Feb/08 09:16 PM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12374902/patch_2194_5.patch
against trunk revision 616796.

@author +1. The patch does not contain any @author tags.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1752/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1752/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1752/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1752/console

This message is automatically generated.


dhruba borthakur added a comment - 07/Feb/08 02:48 PM
I just committed this. Thanks Mahadev!

Hudson added a comment - 08/Feb/08 12:01 PM