[MAPREDUCE-1491] Use HAR filesystem to merge parity files

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.21.0
• Component/s: contrib/raid
• Hadoop Flags: Reviewed
• Assignee: Rodrigo Schmidt
• Reporter: Rodrigo Schmidt
• Created: 14/Feb/10 04:41
• Updated: 24/Aug/10 21:20
• Resolved: 16/Feb/10 23:37

Description

The HDFS raid implementation (HDFS-503) creates a parity file for every file that is RAIDed. This puts additional burden on the memory requirements of the namenode. It will be nice if the parity files are combined together using the HadoopArchive (har) format.

This was (HDFS-684) before, but raid migrated to MAPREDUCE.

Issue Links

• is depended upon by: MAPREDUCE-1492 Delete or recreate obsolete har files used on hdfs raid