Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-73

DFSOutputStream does not close all the sockets

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When DFSOutputStream writes to multiple blocks, it closes only the socket opened for the last block. When it is done with writing to one block it should close the socket.

      I noticed this when I was fixing HADOOP-3067. After fixing HADOOP-3067, there were still a lot of sockets open (but not enough to fail the tests). These sockets were used to write to blocks.

      1. HADOOP-3071.patch
        3 kB
        Raghu Angadi
      2. HDFS-73_0.23_1.patch
        3 kB
        Uma Maheswara Rao G
      3. HDFS-73_0.23.patch
        2 kB
        Uma Maheswara Rao G

        Activity

        Raghu Angadi created issue -
        Raghu Angadi made changes -
        Field Original Value New Value
        Attachment HADOOP-3071.patch [ 12378424 ]
        Raghu Angadi made changes -
        Affects Version/s 0.17.0 [ 12312913 ]
        Fix Version/s 0.16.2 [ 12313051 ]
        Affects Version/s 0.16.0 [ 12312740 ]
        Fix Version/s 0.17.0 [ 12312913 ]
        Priority Blocker [ 1 ] Major [ 3 ]
        Description
        When DFSOutputStream writes to multiple blocks, it closes only the socket opened for the last block. When it is done with writing to one block it should close the socket.

        I noticed this when I was fixing HADOOP-3067. After fixing HADOOP-3067, there were still a lot of sockets open (but not enough to fail the tests). These sockets were used to write to blocks.
        When DFSOutputStream writes to multiple blocks, it closes only the socket opened for the last block. When it is done with writing to one block it should close the socket.

        I noticed this when I was fixing HADOOP-3067. After fixing HADOOP-3067, there were still a lot of sockets open (but not enough to fail the tests). These sockets were used to write to blocks.
        Robert Chansler made changes -
        Fix Version/s 0.17.0 [ 12312913 ]
        Robert Chansler made changes -
        Component/s dfs [ 12310710 ]
        Owen O'Malley made changes -
        Project Hadoop Common [ 12310240 ] HDFS [ 12310942 ]
        Key HADOOP-3071 HDFS-73
        Affects Version/s 0.17.0 [ 12312913 ]
        Component/s dfs [ 12310710 ]
        Eli Collins made changes -
        Fix Version/s 0.23.0 [ 12315571 ]
        Eli Collins made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Uma Maheswara Rao G made changes -
        Attachment HDFS-73_0.23.patch [ 12487580 ]
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Uma Maheswara Rao G made changes -
        Attachment HDFS-73_0.23_1.patch [ 12490270 ]
        Uma Maheswara Rao G made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Uma Maheswara Rao G made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Raghu Angadi [ rangadi ] Uma Maheswara Rao G [ umamaheswararao ]
        Eli Collins made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Eli Collins made changes -
        Component/s hdfs client [ 12312928 ]

          People

          • Assignee:
            Uma Maheswara Rao G
            Reporter:
            Raghu Angadi
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development