Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2316 [umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP
  3. HDFS-2494

[webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.0, 1.0.0
    • Component/s: webhdfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      As part of the reliable test,
      Scenario:
      Initially check the socket count. ---there are aroud 42 sockets are there.
      open the file with DataNode http address using op=OPEN request parameter about 500 times in loop.
      Wait for some time and check the socket count. — There are thousands of ESTABLISHED sockets are growing. ~2052

      Here is the netstat result:

      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042
      C:\Users\uma>netstat | grep 127.0.0.1 | grep ESTABLISHED |wc -l
      2042

      This count is not coming down.

      1. HDFS-2494-20.205Security.patch
        4 kB
        Uma Maheswara Rao G
      2. HDFS-2494.patch
        2 kB
        Uma Maheswara Rao G
      3. HDFS-2494.patch
        5 kB
        Uma Maheswara Rao G
      4. h2494_20111024b.patch
        2 kB
        Tsz Wo Nicholas Sze
      5. h2494_20111024.patch
        1 kB
        Tsz Wo Nicholas Sze

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Uma Maheswara Rao G
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development