Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9336

JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0
    • 3.3.0
    • log-aggregation
    • None
    • Reviewed

    Description

      The JobHistoryServer is leaking CLOSE_WAIT connections to DataNodes whenever viewing a huge log file in JobhistoryServer. This happens only when the below is configured.

      yarn.log-aggregation.file-formats=IndexedFormat
      yarn.log-aggregation.file-controller.IndexedFormat.class=org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController
      yarn.log.server.url=http://jobhistory-host:19888/jobhistory/logs
      

      On investigation, I found that the FSDataInputStream is not closed in org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.IndexedFileAggregatedLogsBlock . Since this block is called every time the Jobhistory page displays the logs, CLOSE_WAIT connections to DataNodes keep on increasing in JobHistoryServer.

       

      Attachments

        1. YARN-9336-branch-2.9.003.patch
          7 kB
          Tarun Parimi
        2. YARN-9336-branch-2.9.002.patch
          7 kB
          Tarun Parimi
        3. YARN-9336-branch-2.9.001.patch
          7 kB
          Tarun Parimi
        4. YARN-9336-branch-2.001.patch
          7 kB
          Tarun Parimi
        5. YARN-9336.002.patch
          4 kB
          Tarun Parimi
        6. YARN-9336.001.patch
          4 kB
          Tarun Parimi

        Activity

          People

            tarunparimi Tarun Parimi
            tarunparimi Tarun Parimi
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: