[YARN-1274] LCE fails to run containers that don't have resources to localize

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.1.1-beta
• Fix Version/s: 2.2.0
• Component/s: nodemanager
• Target Version/s: 2.2.0
• Hadoop Flags: Reviewed
• Assignee: Siddharth Seth
• Reporter: Alejandro Abdelnur
• Created: 05/Oct/13 00:20
• Updated: 30/Jun/15 07:13
• Resolved: 06/Oct/13 00:18

Description

LCE container launch assumes the usercache/USER directory exists and it is owned by the user running the container process.

But the directory is created only if there are resources to localize by the LCE localization command, if there are not resourcdes to localize, LCE localization never executes and launching fails reporting 255 exit code and the NM logs have something like:

2013-10-04 14:07:56,425 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : command provided 1
2013-10-04 14:07:56,425 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : user is llama
2013-10-04 14:07:56,425 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Can't create directory llama in /yarn/nm/usercache/llama/appcache/application_1380853306301_0004/container_1380853306301_0004_01_000004 - Permission denied

Issue Links

• duplicates: YARN-1275 Distributed shell in secure cluster can randomly fail.
• duplicates: YARN-1276 Secure cluster can have random failure to launch a container.