Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1079

HDFS implementation should throw exceptions defined in AbstractFileSystem

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.
      Show
      Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.

      Description

      HDFS implementation Hdfs.java should throw exceptions as defined in AbstractFileSystem. To facilitate this, ClientProtocol should be changed to throw specific exceptions, as defined in AbstractFileSystem.

      1. HDFS-1079.1.patch
        86 kB
        Suresh Srinivas
      2. HDFS-1079.2.patch
        86 kB
        Suresh Srinivas
      3. HDFS-1079.3.patch
        86 kB
        Suresh Srinivas
      4. HDFS-1079.patch
        77 kB
        Suresh Srinivas
      5. HDFS-1079.patch
        78 kB
        Suresh Srinivas

        Issue Links

          Activity

          Suresh Srinivas created issue -
          Suresh Srinivas made changes -
          Field Original Value New Value
          Attachment HDFS-1079.patch [ 12440801 ]
          Hide
          Suresh Srinivas added a comment -

          Updated patch.

          Show
          Suresh Srinivas added a comment - Updated patch.
          Suresh Srinivas made changes -
          Attachment HDFS-1079.patch [ 12440802 ]
          Suresh Srinivas made changes -
          Link This issue relates to HADOOP-6537 [ HADOOP-6537 ]
          Hide
          Sanjay Radia added a comment -
          • dfsclient:
            The methods should declare only IOException - the actual exceptions are declared in Client-protocol; this will make it easier to keep the exception declarations uptodate.
          • FSDirectory - out of sync with trunk
            • Declare more concrete exceptions beyond IOExceptions
              • renameTo -
              • unprotectedRenameTo
              • getPreferredBlockSize
              • addSymlink
          • FSNamesystem
            • Remove IO exception declaration - not thrown
              • unprotectedConcat
              • getBlocklocations*
              • createLocatedBlock
              • concat
              • setTimes
            • Throw more detailed exception
              • startFile*
              • appendFile i
              • delete*
          • File Jira to cleanup IOException - a better exception can be thrown
            • removeBlock, removeLastBlock
            • getCurrentUser
            • plus a few others.
          Show
          Sanjay Radia added a comment - dfsclient: The methods should declare only IOException - the actual exceptions are declared in Client-protocol; this will make it easier to keep the exception declarations uptodate. FSDirectory - out of sync with trunk Declare more concrete exceptions beyond IOExceptions renameTo - unprotectedRenameTo getPreferredBlockSize addSymlink FSNamesystem Remove IO exception declaration - not thrown unprotectedConcat getBlocklocations* createLocatedBlock concat setTimes Throw more detailed exception startFile* appendFile i delete* File Jira to cleanup IOException - a better exception can be thrown removeBlock, removeLastBlock getCurrentUser plus a few others.
          Hide
          Suresh Srinivas added a comment -

          New patch addresses the comments except the following:

          1. FSDirectory - out of sync with trunk
            • Declare more concrete exceptions beyond IOExceptions
              • renameTo - not changing deprecated methods
              • unprotectedRenameTo not changing deprecated methods
          2. FSNamesystem
            • Remove IO exception declaration - not thrown
              • concat - checkPathAccess called from this throws IOException
              • setTimes - method throws IOException
          Show
          Suresh Srinivas added a comment - New patch addresses the comments except the following: FSDirectory - out of sync with trunk Declare more concrete exceptions beyond IOExceptions renameTo - not changing deprecated methods unprotectedRenameTo not changing deprecated methods FSNamesystem Remove IO exception declaration - not thrown concat - checkPathAccess called from this throws IOException setTimes - method throws IOException
          Suresh Srinivas made changes -
          Attachment HDFS-1079.1.patch [ 12443211 ]
          Hide
          Eli Collins added a comment -
          • What tests covers all the new throws of InvalidPathException?
          • Should the rest of h.hdfs.* be converted to HadoopIllegalArgumentException as well?
          Show
          Eli Collins added a comment - What tests covers all the new throws of InvalidPathException? Should the rest of h.hdfs.* be converted to HadoopIllegalArgumentException as well?
          Hide
          Suresh Srinivas added a comment -

          Eli, given we need a test for checking right exceptions are thrown from FileContext for various file systems, I have created Hadoop-6736 to capture that effort. The test is fairly involved and is better off done in a separate jira.

          Yes, we should throw HadoopIllegalArgumentException where currently IllegalArgumentExceptions are thrown.

          Show
          Suresh Srinivas added a comment - Eli, given we need a test for checking right exceptions are thrown from FileContext for various file systems, I have created Hadoop-6736 to capture that effort. The test is fairly involved and is better off done in a separate jira. Yes, we should throw HadoopIllegalArgumentException where currently IllegalArgumentExceptions are thrown.
          Suresh Srinivas made changes -
          Link This issue relates to HDFS-1124 [ HDFS-1124 ]
          Hide
          Sanjay Radia added a comment -

          +1; thanks for addressing my feedback.

          Show
          Sanjay Radia added a comment - +1; thanks for addressing my feedback.
          Hide
          Suresh Srinivas added a comment -

          Updated patch that merges latest trunk changes.

          Show
          Suresh Srinivas added a comment - Updated patch that merges latest trunk changes.
          Suresh Srinivas made changes -
          Attachment HDFS-1079.2.patch [ 12444387 ]
          Suresh Srinivas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Suresh Srinivas made changes -
          Link This issue relates to HADOOP-6736 [ HADOOP-6736 ]
          Suresh Srinivas made changes -
          Link This issue relates to HDFS-1151 [ HDFS-1151 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444387/HDFS-1079.2.patch
          against trunk revision 943306.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444387/HDFS-1079.2.patch against trunk revision 943306. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/358/console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          New patch to fix javadoc warnings

          Show
          Suresh Srinivas added a comment - New patch to fix javadoc warnings
          Suresh Srinivas made changes -
          Attachment HDFS-1079.3.patch [ 12444408 ]
          Suresh Srinivas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Suresh Srinivas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12444408/HDFS-1079.3.patch
          against trunk revision 943306.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12444408/HDFS-1079.3.patch against trunk revision 943306. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/359/console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          Failure of test TestHDFSServerPorts.testBackupNodePorts is unrelated to this patch. This is being tracked in HDFS-887.

          Show
          Suresh Srinivas added a comment - Failure of test TestHDFSServerPorts.testBackupNodePorts is unrelated to this patch. This is being tracked in HDFS-887 .
          Hide
          Suresh Srinivas added a comment -

          Thanks for the review Sanjay. I committed the patch to trunk.

          Show
          Suresh Srinivas added a comment - Thanks for the review Sanjay. I committed the patch to trunk.
          Suresh Srinivas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Release Note Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.
          Resolution Fixed [ 1 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          8h 45m 1 Suresh Srinivas 13/May/10 18:36
          Open Open Patch Available Patch Available
          37d 10h 9m 2 Suresh Srinivas 13/May/10 18:36
          Patch Available Patch Available Resolved Resolved
          10h 4m 1 Suresh Srinivas 14/May/10 04:41
          Resolved Resolved Closed Closed
          577d 2h 36m 1 Konstantin Shvachko 12/Dec/11 06:18

            People

            • Assignee:
              Suresh Srinivas
              Reporter:
              Suresh Srinivas
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development