[YARN-11457] NodeManager Resource Leak when handling a container log with colon

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.3
• Component/s: nodemanager
• Assignee: Vineeth Naroju
• Reporter: Prabhu Joseph
• Created: 16/Mar/23 07:42
• Updated: 16/Mar/23 07:42

Description

NodeManager Leaks the resources when handling a container log with colon. The Illegal file name is not handled and leads to resource leak at NodeManager side.

 

2023-03-14 11:03:53,390 WARN org.apache.hadoop.util.concurrent.ExecutorHelper (ContainersLauncher #2683): Caught exception in thread ContainersLauncher #2683: 
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: taskmanager.log.2023-03-14 09:44-1
	at org.apache.hadoop.fs.Path.initialize(Path.java:263)
	at org.apache.hadoop.fs.Path.<init>(Path.java:221)
	at org.apache.hadoop.fs.Path.<init>(Path.java:129)
	at org.apache.hadoop.fs.Globber.doGlob(Globber.java:270)
	at org.apache.hadoop.fs.Globber.glob(Globber.java:149)
	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2096)
	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2078)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.handleContainerExitWithFailure(ContainerLaunch.java:653)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.handleContainerExitCode(ContainerLaunch.java:593)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:337)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:101)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: taskmanager.log.2023-03-14 09:44-1
	at java.net.URI.checkPath(URI.java:1823)
	at java.net.URI.<init>(URI.java:745)
	at org.apache.hadoop.fs.Path.initialize(Path.java:260)
	... 14 more 

NodeManager status details shows Application stuck in FINISHING_CONTAINER_WAIT, Containers stuck in KILLING state.