[HDFS-72] IOException at task startup "No valid local directories in property: dfs.client.buffer.dir"

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Assignee: Unassigned
• Reporter: Koji Noguchi
• Created: 19/Jun/07 07:27
• Updated: 22/Nov/11 17:06
• Resolved: 22/Nov/11 17:06

Description

When tasks started up on a brand new mapred cluster, I saw couples of them fail immediately by

java.io.IOException: No valid local directories in property: dfs.client.buffer.dir
at org.apache.hadoop.conf.Configuration.getFile(Configuration.java:401)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:1037)
at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1004)
at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:276)
at org.apache.hadoop.dfs.DistributedFileSystem$RawDistributedFileSystem.create(DistributedFileSystem.java:143)
at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.(ChecksumFileSystem.java:367)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:442)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:346)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:253)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.copy(CopyFiles.java:215)
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.map(CopyFiles.java:410)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:175)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445)

(Run on version 0.12.3 + many patches, so the line number could be a little off)

Issue Links

• is duplicated by: HADOOP-776 Task failed with java.io.IOException: No valid local directories in property: dfs.client.buffer.dir
• is related to: HADOOP-1513 A likely race condition between the creation of a directory and checking for its existence in the DiskChecker class