[YARN-3925] ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks.

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.1
• Fix Version/s: 2.8.0, 2.7.2, 2.6.3, 3.0.0-alpha1
• Component/s: nodemanager
• Hadoop Flags: Reviewed
• Assignee: Zhihai Xu
• Reporter: Zhihai Xu
• Created: 14/Jul/15 23:36
• Updated: 06/Jan/17 00:50
• Resolved: 24/Jul/15 22:28

Description

ContainerLogsUtils#getContainerLogFile fails to read files from full disks.
getContainerLogFile depends on LocalDirsHandlerService#getLogPathToRead to get the log file, but LocalDirsHandlerService#getLogPathToRead calls logDirsAllocator.getLocalPathToRead and logDirsAllocator uses configuration YarnConfiguration.NM_LOG_DIRS, which will be updated to not include full disks in LocalDirsHandlerService#checkDirs:

    Configuration conf = getConfig();
    List<String> localDirs = getLocalDirs();
    conf.setStrings(YarnConfiguration.NM_LOCAL_DIRS,
                    localDirs.toArray(new String[localDirs.size()]));
    List<String> logDirs = getLogDirs();
    conf.setStrings(YarnConfiguration.NM_LOG_DIRS,
                      logDirs.toArray(new String[logDirs.size()]));

ContainerLogsUtils#getContainerLogFile is used by NMWebServices#getLogs and ContainerLogsPage.ContainersLogsBlock#render to read the log.

Issue Links

• is related to: HADOOP-12252 LocalDirAllocator should not throw NPE with empty string configuration.