Hadoop Common
  1. Hadoop Common
  2. HADOOP-5146

LocalDirAllocator misses files on the local filesystem

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.19.2, 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For some reason the LocalDirAllocator.getLocaPathToRead doesn't find files which are present, extra logging shows:

      2009-01-30 06:43:32,312 INFO org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: in ifExists, /grid/2/arunc/mapred-local/taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst exists
      2009-01-30 06:43:32,389 WARN org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: in getLocalPathToRead, taskTracker/archive/xxx.yyy.com/tera/in/_partition.lst doesn't exist
      2009-01-30 06:43:32,390 WARN org.apache.hadoop.mapred.TaskRunner: attempt_200901300512_0007_m_000055_0 Child Error
       org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/archive/xx.yyy.com/tera/in/_partition.lst in any of the configured local directories
               at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:388)
               at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
               at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:172)
      
      1. localdirallocator.patch
        2 kB
        Devaraj Das
      2. localdirallocator.patch
        7 kB
        Devaraj Das
      3. 5146.patch
        5 kB
        Devaraj Das
      4. 5146.patch
        2 kB
        Devaraj Das
      5. 5146_20090204job.output.txt
        4 kB
        Tsz Wo Nicholas Sze

        Activity

        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hemanth Yamijala made changes -
        Hadoop Flags [Reviewed]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.19.2 [ 12313650 ]
        Resolution Fixed [ 1 ]
        Amareshwari Sriramadasu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Amareshwari Sriramadasu made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Devaraj Das made changes -
        Component/s mapred [ 12310690 ]
        Devaraj Das made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj Das made changes -
        Attachment 5146.patch [ 12401022 ]
        Devaraj Das made changes -
        Attachment 5146.patch [ 12400740 ]
        Tsz Wo Nicholas Sze made changes -
        Attachment 5146_20090204job.output.txt [ 12399467 ]
        Devaraj Das made changes -
        Assignee Devaraj Das [ devaraj ]
        Devaraj Das made changes -
        Attachment localdirallocator.patch [ 12399202 ]
        Devaraj Das made changes -
        Field Original Value New Value
        Attachment localdirallocator.patch [ 12399099 ]
        Arun C Murthy created issue -

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Arun C Murthy
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development