[YARN-9557] Application fails in diskchecker when ReadWriteDiskValidator is configured.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: nodemanager

Environment

Configure:

<property>
<name>yarn.nodemanager.disk-validator</name>
<value>read-write</value>
</property>

• Assignee: Bilwa S T
• Reporter: Anuruddh Nayak
• Created: 15/May/19 07:05
• Updated: 10/Jul/19 09:21
• Resolved: 10/Jul/19 09:21

Description

Application fails to execute successfully when ReadWriteDiskValidator is configured.

<property>
<name>yarn.nodemanager.disk-validator</name>
<value>read-write</value>
</property>

Exception thrown while starting Container:

java.io.IOException: org.apache.hadoop.util.DiskChecker$DiskErrorException: Disk Check failed!
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:200)
 at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:180)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1233)
 Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Disk Check failed!
 at org.apache.hadoop.util.ReadWriteDiskValidator.checkStatus(ReadWriteDiskValidator.java:82)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.download(ContainerLocalizer.java:255)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.localizeFiles(ContainerLocalizer.java:312)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:198)
 ... 2 more
 Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: /opt/HA/AN0805/nmlocal/usercache/dsperf/appcache/application_1557736108162_0009/filecache/11 is not a directory!
 at org.apache.hadoop.util.ReadWriteDiskValidator.checkStatus(ReadWriteDiskValidator.java:50)