[MAPREDUCE-1831] BlockPlacement policy for RAID

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: contrib/raid
• Hadoop Flags: Reviewed
• Assignee: Scott Chen
• Reporter: Scott Chen
• Created: 01/Jun/10 19:01
• Updated: 15/Nov/11 00:50
• Resolved: 10/Dec/10 22:29

Description

Raid introduce the new dependency between blocks within a file.
The blocks help decode each other. Therefore we should avoid put them on the same machine.

The proposed BlockPlacementPolicy does the following
1. When writing parity blocks, it avoid the parity blocks and source blocks sit together.
2. When reducing replication number, it deletes the blocks that sits with other dependent blocks.
3. It does not change the way we write normal files. It only has different behavior when processing raid files.

Issue Links

• is part of: MAPREDUCE-1969 Allow raid to use Reed-Solomon erasure codes
• is related to: MAPREDUCE-1861 Raid should rearrange the replicas while raiding