Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1513

A likely race condition between the creation of a directory and checking for its existence in the DiskChecker class

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.14.0
    • 0.14.0
    • fs
    • None

    Description

      Got this exception in a job run. It looks like the problem is a race condition between the creation of a directory and checking for its existence. Specifically, the line:
      if (!dir.exists() && !dir.mkdirs()), doesn't seem safe when invoked by multiple processes at the same time.

      2007-06-21 07:55:33,583 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
      2007-06-21 07:55:33,818 WARN org.apache.hadoop.fs.AllocatorPerContext: org.apache.hadoop.util.DiskChecker$DiskErrorException: can not create directory: /export/crawlspace/kryptonite/ddas/dfs/data/tmp
      at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:26)
      at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createPath(LocalDirAllocator.java:211)
      at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:248)
      at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createTmpFileForWrite(LocalDirAllocator.java:276)
      at org.apache.hadoop.fs.LocalDirAllocator.createTmpFileForWrite(LocalDirAllocator.java:155)
      at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.newBackupFile(DFSClient.java:1171)
      at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1136)
      at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:342)
      at org.apache.hadoop.dfs.DistributedFileSystem$RawDistributedFileSystem.create(DistributedFileSystem.java:145)
      at org.apache.hadoop.fs.ChecksumFileSystem$FSOutputSummer.(ChecksumFileSystem.java:368)
      at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:443)
      at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:254)
      at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:675)
      at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:165)
      at org.apache.hadoop.examples.RandomWriter$Map.map(RandomWriter.java:137)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1740)

      2007-06-21 07:55:33,821 WARN org.apache.hadoop.mapred.TaskTracker: Error running child

      Attachments

        1. 1513.patch
          2 kB
          Devaraj Das
        2. 1513.patch
          1 kB
          Devaraj Das
        3. 1513.patch
          0.6 kB
          Devaraj Das
        4. 1513.withjavadocfix.patch
          2 kB
          Devaraj Das

        Issue Links

          Activity

            People

              ddas Devaraj Das
              ddas Devaraj Das
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: