Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1118

DFSOutputStream socket leak when cannot connect to DataNode

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2, 0.20-append, 0.21.0
    • Fix Version/s: 0.20-append, 0.20.205.0, 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The offending code is in DFSOutputStream.nextBlockOutputStream

      This function retries several times to call createBlockOutputStream. Each time when it fails, it leaves a Socket object in DFSOutputStream.s.
      That object is never closed, but overwritten the next time createBlockOutputStream is called.

      1. HDFS-1118.1.patch
        0.5 kB
        Zheng Shao
      2. HDFS-1118.2.patch
        1 kB
        Zheng Shao
      3. hdfs-1118.20s.patch
        3 kB
        Suresh Srinivas
      4. trunkPatch.txt
        1 kB
        dhruba borthakur

        Activity

        Hide
        Matt Foley added a comment -

        Closed upon release of 0.20.205.0

        Show
        Matt Foley added a comment - Closed upon release of 0.20.205.0
        Hide
        Suresh Srinivas added a comment -

        I have committed the patch to 0.20-security branch.

        Show
        Suresh Srinivas added a comment - I have committed the patch to 0.20-security branch.
        Hide
        Jitendra Nath Pandey added a comment -

        +1 for hdfs-1118.20s.patch

        Show
        Jitendra Nath Pandey added a comment - +1 for hdfs-1118.20s.patch
        Hide
        Suresh Srinivas added a comment -

        Merge patch from 0.20-append to 0.20-security

        Show
        Suresh Srinivas added a comment - Merge patch from 0.20-append to 0.20-security
        Hide
        Aaron Baff added a comment -

        This bug also affects version 0.21.0

        Show
        Aaron Baff added a comment - This bug also affects version 0.21.0
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #315 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/315/)
        HDFS-1118. Fix socketleak on DFSClient.
        (Zheng Shao via dhruba)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #315 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/315/ ) HDFS-1118 . Fix socketleak on DFSClient. (Zheng Shao via dhruba)
        Hide
        dhruba borthakur added a comment -

        I just committed this. Thanks Zheng.

        Show
        dhruba borthakur added a comment - I just committed this. Thanks Zheng.
        Hide
        dhruba borthakur added a comment -

        Patch for apache hadop trunk.

        Todd: the finally clause should not hurt, will catch all types of exceptions and cleanup the socket.

        Looks ok without a unit test, because the code is simple and I agree with Zheng that it is difficult to write a unit test

        Show
        dhruba borthakur added a comment - Patch for apache hadop trunk. Todd: the finally clause should not hurt, will catch all types of exceptions and cleanup the socket. Looks ok without a unit test, because the code is simple and I agree with Zheng that it is difficult to write a unit test
        Hide
        dhruba borthakur added a comment -

        Code looks good to me. I will commit this to trunk.

        Show
        dhruba borthakur added a comment - Code looks good to me. I will commit this to trunk.
        Hide
        Todd Lipcon added a comment -

        Marking this for 0.20-append

        Show
        Todd Lipcon added a comment - Marking this for 0.20-append
        Hide
        Todd Lipcon added a comment -

        Hey Zheng. I was just running into this problem and then remembered that you put this patch up a while back.

        Why not just add an IOUtils.closeSocket() call inside the IOException catch clause, instead of adding the finally {} clause?

        Show
        Todd Lipcon added a comment - Hey Zheng. I was just running into this problem and then remembered that you put this patch up a while back. Why not just add an IOUtils.closeSocket() call inside the IOException catch clause, instead of adding the finally {} clause?
        Hide
        Zheng Shao added a comment -

        Moved the cleanup to finally section.

        Show
        Zheng Shao added a comment - Moved the cleanup to finally section.
        Hide
        dhruba borthakur added a comment -

        +1 good catch. awesome!

        Show
        dhruba borthakur added a comment - +1 good catch. awesome!
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12443148/HDFS-1118.1.patch
        against trunk revision 939091.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/165/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443148/HDFS-1118.1.patch against trunk revision 939091. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/165/console This message is automatically generated.
        Hide
        Zheng Shao added a comment -

        This patch does not have a unit test yet. Not sure what is the best way to write one for this.

        Show
        Zheng Shao added a comment - This patch does not have a unit test yet. Not sure what is the best way to write one for this.
        Hide
        Todd Lipcon added a comment -

        Great catch, Zheng! I've seen leaked sockets before but never been able to figure out what the cause was.

        Show
        Todd Lipcon added a comment - Great catch, Zheng! I've seen leaked sockets before but never been able to figure out what the cause was.

          People

          • Assignee:
            Zheng Shao
            Reporter:
            Zheng Shao
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development