Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5146

LocalDirAllocator misses files on the local filesystem

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For some reason the LocalDirAllocator.getLocaPathToRead doesn't find files which are present, extra logging shows:

      2009-01-30 06:43:32,312 INFO org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: in ifExists, /grid/2/arunc/mapred-local/taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst exists
      2009-01-30 06:43:32,389 WARN org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: in getLocalPathToRead, taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst doesn't exist
      2009-01-30 06:43:32,390 WARN org.apache.hadoop.mapred.TaskRunner: attempt_200901300512_0007_m_000055_0 Child Error
       org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/archive/xx.yyy.com/tera/in/_partition.lst in any of the configured local directories
               at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:388)
               at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
               at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:172)
      

        Attachments

        1. 5146.patch
          2 kB
          Devaraj Das
        2. 5146.patch
          5 kB
          Devaraj Das
        3. 5146_20090204job.output.txt
          4 kB
          Tsz Wo Nicholas Sze
        4. localdirallocator.patch
          7 kB
          Devaraj Das
        5. localdirallocator.patch
          2 kB
          Devaraj Das

          Activity

            People

            • Assignee:
              devaraj Devaraj Das
              Reporter:
              acmurthy Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: