[YARN-7498] NM failed to start if the namespace of remote log dirs differs from fs.defaultFS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: sandflee
• Reporter: sandflee
• Created: 15/Nov/17 03:54
• Updated: 15/Nov/17 06:34
• Resolved: 15/Nov/17 06:32

Description

fs.defaultFS is hdfs://nameservice1 and yarn.nodemanager.remote-app-log-dir is hdfs://nameservice2, when nm start see errors:

java.lang.IllegalArgumentException: Wrong FS: hdfs://nameservice2/yarn-logs, expected: hdfs://nameservice1
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1128)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1124)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1124)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:192)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:319)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:443)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:67)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
at java.lang.Thread.run(Thread.java:745)

Issue Links

• duplicates: YARN-3269 Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path