Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3359

DFSClient.close should close cached sockets

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      Some applications like the TT/JT (pre-2.0) and probably the RM/NM cycle through DistributedFileSystem objects reasonably frequently. So long as they call close() it isn't a big problem, except that currently DFSClient.close() doesn't explicitly close the SocketCache. So unless a full GC runs (causing the references to get finalized), many SocketCaches can get orphaned, each with many open sockets inside. We should fix the close() function to close all cached sockets.

      1. hdfs-3359-branch-0.23.txt
        2 kB
        Robert Joseph Evans
      2. hdfs-3359.txt
        3 kB
        Todd Lipcon
      3. hdfs-3359.txt
        3 kB
        Todd Lipcon

        Issue Links

          Activity

          Todd Lipcon created issue -
          Todd Lipcon made changes -
          Field Original Value New Value
          Attachment hdfs-3359.txt [ 12525482 ]
          Todd Lipcon made changes -
          Attachment hdfs-3359.txt [ 12525483 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Link This issue relates to HDFS-941 [ HDFS-941 ]
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.0.0 [ 12320353 ]
          Resolution Fixed [ 1 ]
          Robert Joseph Evans made changes -
          Attachment hdfs-3359-branch-0.23.txt [ 12525628 ]
          Todd Lipcon made changes -
          Fix Version/s 0.23.3 [ 12320052 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HDFS-3373 [ HDFS-3373 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.0.0-alpha [ 12320353 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Colin Patrick McCabe made changes -
          Link This issue relates to HDFS-7718 [ HDFS-7718 ]

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development