Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1836

Thousand of CLOSE_WAIT socket

        Details

        • Type: Bug Bug
        • Status: Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: 0.20.2
        • Fix Version/s: 0.20.3, 0.20.205.0
        • Component/s: hdfs-client
        • Labels:
          None
        • Environment:

          Linux 2.6.18-194.32.1.el5 #1 SMP Wed Jan 5 17:52:25 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
          java version "1.6.0_23"
          Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
          Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)

        • Hadoop Flags:
          Reviewed

          Description

          $ /usr/sbin/lsof -i TCP:50010 | grep -c CLOSE_WAIT
          4471

          It is better if everything runs normal.
          However, from time to time there are some "DataStreamer Exception: java.net.SocketTimeoutException" and "DFSClient.processDatanodeError(2507) | Error Recovery for" can be found from log file and the number of CLOSE_WAIT socket just keep increasing

          The CLOSE_WAIT handles may remain for hours and days; then "Too many open file" some day.

          1. Text File
            patch-draft-1836.patch
            2 kB
            Dennis Cheung
          2. Text File
            hdfs-1836-0.20.txt
            2 kB
            Todd Lipcon
          3. Text File
            hdfs-1836-0.20.txt
            2 kB
            Todd Lipcon
          4. Text File
            hdfs-1836-0.20.205.txt
            1 kB
            Bharath Mundlapudi

            Activity

            Descending order - Click to sort in ascending order
            Matt Foley made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            Matt Foley made changes -
            Fix Version/s 0.20.205.0 [ 12316392 ]
            Fix Version/s 0.20.204.0 [ 12316319 ]
            Owen O'Malley made changes -
            Fix Version/s 0.20.205.0 [ 12316392 ]
            Owen O'Malley made changes -
            Fix Version/s 0.20.204.0 [ 12316319 ]
            Matt Foley made changes -
            Fix Version/s 0.20.205.0 [ 12316392 ]
            Matt Foley made changes -
            Status Reopened [ 4 ] Resolved [ 5 ]
            Fix Version/s 0.20.205.0 [ 12316392 ]
            Resolution Fixed [ 1 ]
            Matt Foley made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            Assignee Todd Lipcon [ tlipcon ] Bharath Mundlapudi [ bharathm ]
            Bharath Mundlapudi made changes -
            Attachment hdfs-1836-0.20.205.txt [ 12480703 ]
            Bharath Mundlapudi made changes -
            Fix Version/s 0.20.205.0 [ 12316392 ]
            Eli Collins made changes -
            Status Patch Available [ 10002 ] Resolved [ 5 ]
            Hadoop Flags [Reviewed]
            Fix Version/s 0.20.3 [ 12314814 ]
            Resolution Fixed [ 1 ]
            Eli Collins made changes -
            Assignee Todd Lipcon [ tlipcon ]
            Todd Lipcon made changes -
            Attachment hdfs-1836-0.20.txt [ 12479218 ]
            Todd Lipcon made changes -
            Attachment hdfs-1836-0.20.txt [ 12479216 ]
            Dennis Cheung made changes -
            Attachment patch-draft-1836.patch [ 12478378 ]
            Dennis Cheung made changes -
            Field Original Value New Value
            Status Open [ 1 ] Patch Available [ 10002 ]
            Dennis Cheung created issue -

              People

              • Assignee:
                Bharath Mundlapudi
                Reporter:
                Dennis Cheung
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development