Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2352

RAID blockfixer can use a heuristic to find unfixable files

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Won't Fix
    • None
    • None
    • contrib/raid
    • None

    Description

      It is possible to have corrupt files that were never RAIDed. In such a case, there is no use in trying to submit a block fixer job for that file. The RAID code has the function filterUnfixableSourceFiles() that checks for the presence of parity files for each source file. This is too expensive, since a lot of the parity files can be HARed. Instead, we can use a heuristic where we just check for the presence of the parent directory in the parity space. If the parent directory is absent, the parity file cannot be present, and the source file would be unfixable.

      Attachments

        Activity

          People

            rvadali Ramkumar Vadali
            rvadali Ramkumar Vadali
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: