[HADOOP-2154] Non-interleaved checksums would optimize block transfers.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 0.18.0
• Assignee: Raghu Angadi
• Reporter: Konstantin Shvachko
• Created: 06/Nov/07 01:54
• Updated: 08/Jul/09 16:42
• Resolved: 14/May/08 06:39

Description

Currently when a block is transfered to a data-node the client interleaves data chunks with the respective checksums.
This requires creating an extra copy of the original data in a new buffer interleaved with the crcs.
We can avoid extra copying if the data and the crc are fed to the socket one after another.

Issue Links

• is related to: HADOOP-1702 Reduce buffer copies when data is written to DFS
• is related to: HADOOP-2758 Reduce memory copies when data is read from DFS