Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1145

Potential file handle leak in aggregated logs web ui

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.5-alpha, 0.23.9, 2.1.1-beta
    • Fix Version/s: 0.23.11, 2.3.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Any problem in getting aggregated logs for rendering on web ui, then LogReader is not closed.

      Now, it reader is not closed which causing many connections in close_wait state.

      hadoopuser@hadoopuser:> jps
      27909 JobHistoryServer

      DataNode port is 50010. When greped with DataNode port, many connections are in CLOSE_WAIT from JHS.
      hadoopuser@hadoopuser:> netstat -tanlp |grep 50010
      tcp 0 0 10.18.40.48:50010 0.0.0.0:* LISTEN 21453/java
      tcp 1 0 10.18.40.48:20596 10.18.40.48:50010 CLOSE_WAIT 27909/java
      tcp 1 0 10.18.40.48:19667 10.18.40.152:50010 CLOSE_WAIT 27909/java
      tcp 1 0 10.18.40.48:20593 10.18.40.48:50010 CLOSE_WAIT 27909/java
      tcp 1 0 10.18.40.48:12290 10.18.40.48:50010 CLOSE_WAIT 27909/java
      tcp 1 0 10.18.40.48:19662 10.18.40.152:50010 CLOSE_WAIT 27909/java

        Attachments

        1. MAPREDUCE-5486.patch
          2 kB
          Rohith Sharma K S
        2. YARN-1145.patch
          2 kB
          Rohith Sharma K S
        3. YARN-1145.1.patch
          3 kB
          Rohith Sharma K S
        4. YARN-1145.2.patch
          3 kB
          Rohith Sharma K S
        5. YARN-1145.3.patch
          3 kB
          Rohith Sharma K S
        6. YARN-1145.4.patch
          12 kB
          Rohith Sharma K S

          Activity

            People

            • Assignee:
              rohithsharma Rohith Sharma K S
              Reporter:
              rohithsharma Rohith Sharma K S
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: