Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3925

ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks.

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      ContainerLogsUtils#getContainerLogFile fails to read files from full disks.
      getContainerLogFile depends on LocalDirsHandlerService#getLogPathToRead to get the log file, but LocalDirsHandlerService#getLogPathToRead calls logDirsAllocator.getLocalPathToRead and logDirsAllocator uses configuration YarnConfiguration.NM_LOG_DIRS, which will be updated to not include full disks in LocalDirsHandlerService#checkDirs:

          Configuration conf = getConfig();
          List<String> localDirs = getLocalDirs();
          conf.setStrings(YarnConfiguration.NM_LOCAL_DIRS,
                          localDirs.toArray(new String[localDirs.size()]));
          List<String> logDirs = getLogDirs();
          conf.setStrings(YarnConfiguration.NM_LOG_DIRS,
                            logDirs.toArray(new String[logDirs.size()]));
      

      ContainerLogsUtils#getContainerLogFile is used by NMWebServices#getLogs and ContainerLogsPage.ContainersLogsBlock#render to read the log.

      Attachments

        1. YARN-3925.000.patch
          11 kB
          Zhihai Xu
        2. YARN-3925.001.patch
          7 kB
          Zhihai Xu

        Issue Links

          Activity

            People

              zxu Zhihai Xu
              zxu Zhihai Xu
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: