Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5581

libhdfs does not get FileNotFoundException

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.19.1
        • Fix Version/s: 0.21.0
        • Component/s: None
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          When trying to open a file that does not exist for read, libhdfs prints an error out along the line of "Cannot open file <filename>".

          I believe it should be throwing the FileNotFoundException instead. This would allow us to correctly set the errno and more naturally give the errors to the C-based clients.

          1. Text File
            HADOOP-5581.patch
            3 kB
            Brian Bockelman
          2. Text File
            HADOOP-5581-v2.patch
            2 kB
            Brian Bockelman

            Issue Links

            is depended upon by

            Bug - A problem which impairs or prevents the functions of the product. HADOOP-5579 libhdfs does not set errno correctly

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              craigm Craig Macdonald added a comment -

              Hi Brian, Two small questions:
              (a) Can you comment on how this relates to HADOOP-5579?
              (b) I can't see "Cannot open file" in the current trunk of hdfs.c - do you know where this is coming from?

              Show
              craigm Craig Macdonald added a comment - Hi Brian, Two small questions: (a) Can you comment on how this relates to HADOOP-5579 ? (b) I can't see "Cannot open file" in the current trunk of hdfs.c - do you know where this is coming from?
              Hide
              Permalink
              bockelman Brian Bockelman added a comment -

              This patch causes HDFS to throw a FileNotFoundException in FSNamesystem instead of a generic IOException.

              HADOOP-5579 fixes the parsing of the FileNotFoundException; with this patch, it should propagate the errno correctly.

              Show
              bockelman Brian Bockelman added a comment - This patch causes HDFS to throw a FileNotFoundException in FSNamesystem instead of a generic IOException. HADOOP-5579 fixes the parsing of the FileNotFoundException; with this patch, it should propagate the errno correctly.
              Hide
              Permalink
              hadoopqa Hadoop QA added a comment -

              -1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12404071/HADOOP-5581.patch
              against trunk revision 759398.

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 3 new or modified tests.

              +1 javadoc. The javadoc tool did not generate any warning messages.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 findbugs. The patch does not introduce any new Findbugs warnings.

              +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              +1 core tests. The patch passed core unit tests.

              -1 contrib tests. The patch failed contrib unit tests.

              Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/testReport/
              Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
              Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/checkstyle-errors.html
              Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/console

              This message is automatically generated.

              Show
              hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12404071/HADOOP-5581.patch against trunk revision 759398. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/77/console This message is automatically generated.
              Hide
              Permalink
              rangadi Raghu Angadi added a comment -

              Patch looks fine. A few improvements :

              JavaDoc for getBlockLocations could be modifed to make this a part of the contract.
              Do we need any text for FNF excepton?

              Instead of adding a new test, a test simple test case could be added to another test (e.g. TestDistributedFIleSystem.java). That way it comes almost for free (see test added in HADOOP-5191). Each new test adds on the order of 7-10 seconds, however simple it is.

              Show
              rangadi Raghu Angadi added a comment - Patch looks fine. A few improvements : JavaDoc for getBlockLocations could be modifed to make this a part of the contract. Do we need any text for FNF excepton? Instead of adding a new test, a test simple test case could be added to another test (e.g. TestDistributedFIleSystem.java). That way it comes almost for free (see test added in HADOOP-5191 ). Each new test adds on the order of 7-10 seconds, however simple it is.
              Hide
              Permalink
              bockelman Brian Bockelman added a comment -

              Patch updated - made a note in the javadoc that getBlockLocation throws FNF. Used TestDistributedFileSystem class instead of a new test class.

              Show
              bockelman Brian Bockelman added a comment - Patch updated - made a note in the javadoc that getBlockLocation throws FNF. Used TestDistributedFileSystem class instead of a new test class.
              Hide
              Permalink
              rangadi Raghu Angadi added a comment -

              +1.

              Regd the contract, it needs to be added to ClientProtocol.getBlockLocations() and FNF is not the only exception. It is ok for now.

              Show
              rangadi Raghu Angadi added a comment - +1. Regd the contract, it needs to be added to ClientProtocol.getBlockLocations() and FNF is not the only exception. It is ok for now.
              Hide
              Permalink
              hadoopqa Hadoop QA added a comment -

              -1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12404491/HADOOP-5581-v2.patch
              against trunk revision 763728.

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 3 new or modified tests.

              +1 javadoc. The javadoc tool did not generate any warning messages.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 findbugs. The patch does not introduce any new Findbugs warnings.

              +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              -1 core tests. The patch failed core unit tests.

              -1 contrib tests. The patch failed contrib unit tests.

              Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/testReport/
              Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
              Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/checkstyle-errors.html
              Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/console

              This message is automatically generated.

              Show
              hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12404491/HADOOP-5581-v2.patch against trunk revision 763728. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/179/console This message is automatically generated.
              Hide
              Permalink
              rangadi Raghu Angadi added a comment -

              I just committed this to trunk. Thanks Brian.

              This is not marked a blocker for 0.19 and 0.20.

              Show
              rangadi Raghu Angadi added a comment - I just committed this to trunk. Thanks Brian. This is not marked a blocker for 0.19 and 0.20.
              Hide
              Permalink
              hudson Hudson added a comment -

              Integrated in Hadoop-trunk #811 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/811/)

              Show
              hudson Hudson added a comment - Integrated in Hadoop-trunk #811 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/811/ )

                People

                • Assignee:
                  bockelman Brian Bockelman
                  Reporter:
                  bockelman Brian Bockelman
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development