Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2352

RAID blockfixer can use a heuristic to find unfixable files

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: contrib/raid
    • Labels:
      None

      Description

      It is possible to have corrupt files that were never RAIDed. In such a case, there is no use in trying to submit a block fixer job for that file. The RAID code has the function filterUnfixableSourceFiles() that checks for the presence of parity files for each source file. This is too expensive, since a lot of the parity files can be HARed. Instead, we can use a heuristic where we just check for the presence of the parent directory in the parity space. If the parent directory is absent, the parity file cannot be present, and the source file would be unfixable.

        Attachments

          Activity

            People

            • Assignee:
              rvadali Ramkumar Vadali
              Reporter:
              rvadali Ramkumar Vadali
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: