[YARN-5373] NPE listing wildcard directory in containerLaunch

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.9.0
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: nodemanager
• Hadoop Flags: Reviewed
• Assignee: Daniel Templeton
• Reporter: Haibo Chen
• Created: 13/Jul/16 19:43
• Updated: 23/Jan/17 09:03
• Resolved: 26/Aug/16 18:12

Description

YARN-4958 added support for wildcards in file localization. It introduces a NPE
at

for (File wildLink : directory.listFiles()) {
    sb.symlink(new Path(wildLink.toString()), new Path(wildLink.getName()));
}

When directory.listFiles returns null (only happens in a secure cluster), NPE will cause the container fail to launch.
Hive, Oozie jobs fail as a result.

Issue Links

• is broken by: YARN-4958 The file localization process should allow for wildcards to reduce the application footprint in the state store
• is depended upon by: YARN-5381 fix NPE listing wildcard directory on Windows
• is depended upon by: YARN-5388 Deprecate and remove DockerContainerExecutor
• links to: YARN-4958