Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7551

LocalDirAllocator should incorporate LocalStorage

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.20.204.0
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      The o.a.h.fs.LocalDirAllocator is not aware of o.a.h.m.t.LocalStorage (introduced in MAPREDUCE-2413) - it always considers the configured local dirs, not just the ones that happen to be good. Therefore if there's a disk failure then every call to get a local path will result in LocalDirAllocator#confChanged doing a disk check of all the configured local dirs. It seems like LocalStorage should be a private class to LocalAllocator so that all users of LocalDirAllocator benefit from the disk failure handling and all the various users of LocalDirAllocator don't have to be modified to handle disk failures. Note that LocalDirAllocator already handles faulty directories.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                eli Eli Collins
                Reporter:
                eli Eli Collins
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: