Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1615

seek() on closed DFS input stream throws NPE

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      After closing an input stream on DFS, seeking slightly ahead of the last read will throw an NPE:

      java.lang.NullPointerException
      at org.apache.hadoop.hdfs.DFSInputStream.seek(DFSInputStream.java:749)
      at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:42)

        Activity

        Hide
        M. C. Srivas added a comment -

        >After closing an input stream on DFS, seeking slightly ahead of the last read will throw an NPE:

        Isn't this a good thing? It exposes bugs at the layer above DFS. I'd prefer to keep this behaviour rather than fix it.

        Show
        M. C. Srivas added a comment - >After closing an input stream on DFS, seeking slightly ahead of the last read will throw an NPE: Isn't this a good thing? It exposes bugs at the layer above DFS. I'd prefer to keep this behaviour rather than fix it.
        Hide
        Todd Lipcon added a comment -

        It should throw IOE, not NPE

        Show
        Todd Lipcon added a comment - It should throw IOE, not NPE
        Hide
        Scott Carey added a comment -

        Patch contains simple fix, and unit test that fails without the fix.

        Show
        Scott Carey added a comment - Patch contains simple fix, and unit test that fails without the fix.
        Hide
        Todd Lipcon added a comment -

        +1 pending Hudson test results

        Show
        Todd Lipcon added a comment - +1 pending Hudson test results
        Hide
        Harsh J added a comment -

        Patch that makes closed inp-stream throw IOE instead. And a new test case for DFSInputStream (I could not find any existing one, so let me know if there is indeed one that covers general open/read/close stuff - I will update patch gladly.)

        Show
        Harsh J added a comment - Patch that makes closed inp-stream throw IOE instead. And a new test case for DFSInputStream (I could not find any existing one, so let me know if there is indeed one that covers general open/read/close stuff - I will update patch gladly.)
        Hide
        Harsh J added a comment -

        Ooh! Darn I did not notice it got re-assigned, sorry 'bout that. Carry on Scott!

        Show
        Harsh J added a comment - Ooh! Darn I did not notice it got re-assigned, sorry 'bout that. Carry on Scott!
        Hide
        Harsh J added a comment -

        Ack, Scott, can you resubmit your file? I believe mine would end up conflicting the Hudson build. I'm unable to figure out how to delete it. Sorry again for not noticing the reassign man!

        Show
        Harsh J added a comment - Ack, Scott, can you resubmit your file? I believe mine would end up conflicting the Hudson build. I'm unable to figure out how to delete it. Sorry again for not noticing the reassign man!
        Hide
        Harsh J added a comment -

        Ah ok, I deleted my patch. My patch was this anyway, for reference: https://github.com/QwertyManiac/hadoop-hdfs/commit/fc7ac93ed5428651b63c8a6315cf6815303b4222

        Show
        Harsh J added a comment - Ah ok, I deleted my patch. My patch was this anyway, for reference: https://github.com/QwertyManiac/hadoop-hdfs/commit/fc7ac93ed5428651b63c8a6315cf6815303b4222
        Hide
        Scott Carey added a comment -

        No problem – JIRA race condition. Your error message string is better than mine. I could not find any more specific testing around this either, but put the new test in an already existing class.

        Show
        Scott Carey added a comment - No problem – JIRA race condition. Your error message string is better than mine. I could not find any more specific testing around this either, but put the new test in an already existing class.
        Hide
        Scott Carey added a comment -

        Actually, I think it was my fault – I thought it was unassigned when I assigned it to me. Sorry!

        Show
        Scott Carey added a comment - Actually, I think it was my fault – I thought it was unassigned when I assigned it to me. Sorry!
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478861/HDFS-1615.patch
        against trunk revision 1102005.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.cli.TestHDFSCLI
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFileConcurrentReader
        org.apache.hadoop.tools.TestJMXGet

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478861/HDFS-1615.patch against trunk revision 1102005. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/484//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478862/HDFS-1615.r1.diff
        against trunk revision 1102005.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.cli.TestHDFSCLI
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFileConcurrentReader
        org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
        org.apache.hadoop.tools.TestJMXGet

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//testReport/
        Release audit warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478862/HDFS-1615.r1.diff against trunk revision 1102005. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//testReport/ Release audit warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/483//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        +1. Committed Scott's patch to trunk and branch-0.22. Thanks Scott and Harsh!

        Show
        Todd Lipcon added a comment - +1. Committed Scott's patch to trunk and branch-0.22. Thanks Scott and Harsh!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #658 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/658/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #658 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/658/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #49 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/49/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/ )

          People

          • Assignee:
            Scott Carey
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development