Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1079

HDFS implementation should throw exceptions defined in AbstractFileSystem

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.
      Show
      Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.

      Description

      HDFS implementation Hdfs.java should throw exceptions as defined in AbstractFileSystem. To facilitate this, ClientProtocol should be changed to throw specific exceptions, as defined in AbstractFileSystem.

      1. HDFS-1079.3.patch
        86 kB
        Suresh Srinivas
      2. HDFS-1079.2.patch
        86 kB
        Suresh Srinivas
      3. HDFS-1079.1.patch
        86 kB
        Suresh Srinivas
      4. HDFS-1079.patch
        77 kB
        Suresh Srinivas
      5. HDFS-1079.patch
        78 kB
        Suresh Srinivas

        Issue Links

          Activity

          Hide
          Suresh Srinivas added a comment -

          Updated patch.

          Show
          Suresh Srinivas added a comment - Updated patch.
          Hide
          Sanjay Radia added a comment -
          • dfsclient:
            The methods should declare only IOException - the actual exceptions are declared in Client-protocol; this will make it easier to keep the exception declarations uptodate.
          • FSDirectory - out of sync with trunk
            • Declare more concrete exceptions beyond IOExceptions
              • renameTo -
              • unprotectedRenameTo
              • getPreferredBlockSize
              • addSymlink
          • FSNamesystem
            • Remove IO exception declaration - not thrown
              • unprotectedConcat
              • getBlocklocations*
              • createLocatedBlock
              • concat
              • setTimes
            • Throw more detailed exception
              • startFile*
              • appendFile i
              • delete*
          • File Jira to cleanup IOException - a better exception can be thrown
            • removeBlock, removeLastBlock
            • getCurrentUser
            • plus a few others.
          Show
          Sanjay Radia added a comment - dfsclient: The methods should declare only IOException - the actual exceptions are declared in Client-protocol; this will make it easier to keep the exception declarations uptodate. FSDirectory - out of sync with trunk Declare more concrete exceptions beyond IOExceptions renameTo - unprotectedRenameTo getPreferredBlockSize addSymlink FSNamesystem Remove IO exception declaration - not thrown unprotectedConcat getBlocklocations* createLocatedBlock concat setTimes Throw more detailed exception startFile* appendFile i delete* File Jira to cleanup IOException - a better exception can be thrown removeBlock, removeLastBlock getCurrentUser plus a few others.
          Hide
          Suresh Srinivas added a comment -

          New patch addresses the comments except the following:

          1. FSDirectory - out of sync with trunk
            • Declare more concrete exceptions beyond IOExceptions
              • renameTo - not changing deprecated methods
              • unprotectedRenameTo not changing deprecated methods
          2. FSNamesystem
            • Remove IO exception declaration - not thrown
              • concat - checkPathAccess called from this throws IOException
              • setTimes - method throws IOException
          Show
          Suresh Srinivas added a comment - New patch addresses the comments except the following: FSDirectory - out of sync with trunk Declare more concrete exceptions beyond IOExceptions renameTo - not changing deprecated methods unprotectedRenameTo not changing deprecated methods FSNamesystem Remove IO exception declaration - not thrown concat - checkPathAccess called from this throws IOException setTimes - method throws IOException
          Hide
          Eli Collins added a comment -
          • What tests covers all the new throws of InvalidPathException?
          • Should the rest of h.hdfs.* be converted to HadoopIllegalArgumentException as well?
          Show
          Eli Collins added a comment - What tests covers all the new throws of InvalidPathException? Should the rest of h.hdfs.* be converted to HadoopIllegalArgumentException as well?
          Hide
          Suresh Srinivas added a comment -

          Eli, given we need a test for checking right exceptions are thrown from FileContext for various file systems, I have created Hadoop-6736 to capture that effort. The test is fairly involved and is better off done in a separate jira.

          Yes, we should throw HadoopIllegalArgumentException where currently IllegalArgumentExceptions are thrown.

          Show
          Suresh Srinivas added a comment - Eli, given we need a test for checking right exceptions are thrown from FileContext for various file systems, I have created Hadoop-6736 to capture that effort. The test is fairly involved and is better off done in a separate jira. Yes, we should throw HadoopIllegalArgumentException where currently IllegalArgumentExceptions are thrown.
          Hide
          Sanjay Radia added a comment -

          +1; thanks for addressing my feedback.

          Show
          Sanjay Radia added a comment - +1; thanks for addressing my feedback.
          Hide
          Suresh Srinivas added a comment -

          Updated patch that merges latest trunk changes.

          Show
          Suresh Srinivas added a comment - Updated patch that merges latest trunk changes.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444387/HDFS-1079.2.patch
          against trunk revision 943306.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444387/HDFS-1079.2.patch against trunk revision 943306. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          New patch to fix javadoc warnings

          Show
          Suresh Srinivas added a comment - New patch to fix javadoc warnings
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444408/HDFS-1079.3.patch
          against trunk revision 943306.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444408/HDFS-1079.3.patch against trunk revision 943306. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          Failure of test TestHDFSServerPorts.testBackupNodePorts is unrelated to this patch. This is being tracked in HDFS-887.

          Show
          Suresh Srinivas added a comment - Failure of test TestHDFSServerPorts.testBackupNodePorts is unrelated to this patch. This is being tracked in HDFS-887 .
          Hide
          Suresh Srinivas added a comment -

          Thanks for the review Sanjay. I committed the patch to trunk.

          Show
          Suresh Srinivas added a comment - Thanks for the review Sanjay. I committed the patch to trunk.

            People

            • Assignee:
              Suresh Srinivas
              Reporter:
              Suresh Srinivas
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development