Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-11039

LogAggregationFileControllerFactory::getFileControllerForRead can leak threads

    XMLWordPrintableJSON

Details

    Description

      getFileControllerForRead::getFileControllerForRead internally opens up a new FS object everytime and is not closed.

      When cloud connectors (e.g s3a) is used along with Knox, it ends up leaking KnoxTokenMonitor for every unclosed FS object causing thread leaks in NM.

      Lines of interest:

      https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/LogAggregationFileControllerFactory.java#L167

             try {
                Path remoteAppLogDir = fileController.getOlderRemoteAppLogDir(appId,
                    appOwner);
                if (LogAggregationUtils.getNodeFiles(conf, remoteAppLogDir, appId,
                    appOwner).hasNext()) {
                  return fileController;
                }
              } catch (Exception ex) {
                diagnosticsMsg.append(ex.getMessage() + "\n");
                continue;
              }
      

      https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/LogAggregationUtils.java#L252

      	
        public static RemoteIterator<FileStatus> getNodeFiles(Configuration conf,
            Path remoteAppLogDir, ApplicationId appId, String appOwner)
            throws IOException {
          Path qualifiedLogDir =
              FileContext.getFileContext(conf).makeQualified(remoteAppLogDir);
          return FileContext.getFileContext(
              qualifiedLogDir.toUri(), conf).listStatus(remoteAppLogDir);
        }
      

      Attachments

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: