Hadoop Common
  1. Hadoop Common
  2. HADOOP-5581

libhdfs does not get FileNotFoundException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When trying to open a file that does not exist for read, libhdfs prints an error out along the line of "Cannot open file <filename>".

      I believe it should be throwing the FileNotFoundException instead. This would allow us to correctly set the errno and more naturally give the errors to the C-based clients.

      1. HADOOP-5581-v2.patch
        2 kB
        Brian Bockelman
      2. HADOOP-5581.patch
        3 kB
        Brian Bockelman

        Issue Links

          Activity

          Brian Bockelman created issue -
          Chris Douglas made changes -
          Field Original Value New Value
          Link This issue blocks HADOOP-5579 [ HADOOP-5579 ]
          Hide
          Craig Macdonald added a comment -

          Hi Brian, Two small questions:
          (a) Can you comment on how this relates to HADOOP-5579?
          (b) I can't see "Cannot open file" in the current trunk of hdfs.c - do you know where this is coming from?

          Show
          Craig Macdonald added a comment - Hi Brian, Two small questions: (a) Can you comment on how this relates to HADOOP-5579 ? (b) I can't see "Cannot open file" in the current trunk of hdfs.c - do you know where this is coming from?
          Hide
          Brian Bockelman added a comment -

          This patch causes HDFS to throw a FileNotFoundException in FSNamesystem instead of a generic IOException.

          HADOOP-5579 fixes the parsing of the FileNotFoundException; with this patch, it should propagate the errno correctly.

          Show
          Brian Bockelman added a comment - This patch causes HDFS to throw a FileNotFoundException in FSNamesystem instead of a generic IOException. HADOOP-5579 fixes the parsing of the FileNotFoundException; with this patch, it should propagate the errno correctly.
          Brian Bockelman made changes -
          Attachment HADOOP-5581.patch [ 12404071 ]
          Brian Bockelman made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.19.2 [ 12313650 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12404071/HADOOP-5581.patch
          against trunk revision 759398.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12404071/HADOOP-5581.patch against trunk revision 759398. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/console This message is automatically generated.
          Hide
          Raghu Angadi added a comment -

          Patch looks fine. A few improvements :

          JavaDoc for getBlockLocations could be modifed to make this a part of the contract.
          Do we need any text for FNF excepton?

          Instead of adding a new test, a test simple test case could be added to another test (e.g. TestDistributedFIleSystem.java). That way it comes almost for free (see test added in HADOOP-5191). Each new test adds on the order of 7-10 seconds, however simple it is.

          Show
          Raghu Angadi added a comment - Patch looks fine. A few improvements : JavaDoc for getBlockLocations could be modifed to make this a part of the contract. Do we need any text for FNF excepton? Instead of adding a new test, a test simple test case could be added to another test (e.g. TestDistributedFIleSystem.java). That way it comes almost for free (see test added in HADOOP-5191 ). Each new test adds on the order of 7-10 seconds, however simple it is.
          Hide
          Brian Bockelman added a comment -

          Patch updated - made a note in the javadoc that getBlockLocation throws FNF. Used TestDistributedFileSystem class instead of a new test class.

          Show
          Brian Bockelman added a comment - Patch updated - made a note in the javadoc that getBlockLocation throws FNF. Used TestDistributedFileSystem class instead of a new test class.
          Brian Bockelman made changes -
          Attachment HADOOP-5581-v2.patch [ 12404491 ]
          Hide
          Raghu Angadi added a comment -

          +1.

          Regd the contract, it needs to be added to ClientProtocol.getBlockLocations() and FNF is not the only exception. It is ok for now.

          Show
          Raghu Angadi added a comment - +1. Regd the contract, it needs to be added to ClientProtocol.getBlockLocations() and FNF is not the only exception. It is ok for now.
          dhruba borthakur made changes -
          Assignee Brian Bockelman [ bockelman ]
          Raghu Angadi made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Raghu Angadi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12404491/HADOOP-5581-v2.patch
          against trunk revision 763728.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12404491/HADOOP-5581-v2.patch against trunk revision 763728. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/console This message is automatically generated.
          Hide
          Raghu Angadi added a comment -

          I just committed this to trunk. Thanks Brian.

          This is not marked a blocker for 0.19 and 0.20.

          Show
          Raghu Angadi added a comment - I just committed this to trunk. Thanks Brian. This is not marked a blocker for 0.19 and 0.20.
          Raghu Angadi made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.0 [ 12313563 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Fix Version/s 0.19.2 [ 12313650 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #811 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/811/ )
          Owen O'Malley made changes -
          Component/s dfs [ 12310710 ]
          Owen O'Malley made changes -
          Component/s libhdfs [ 12311345 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks HADOOP-5579 [ HADOOP-5579 ]
          Gavin made changes -
          Link This issue is depended upon by HADOOP-5579 [ HADOOP-5579 ]

            People

            • Assignee:
              Brian Bockelman
              Reporter:
              Brian Bockelman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development