[HDFS-3429] DataNode reads checksums even if client does not need them

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.0.3-alpha
• Component/s: datanode, performance
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 16/May/12 06:10
• Updated: 03/Sep/14 23:09
• Resolved: 14/Jan/13 20:47

Description

Currently, even if the client does not want to verify checksums, the datanode reads them anyway and sends them over the wire. This means that performance improvements like HBase's application-level checksums don't have much benefit when reading through the datanode, since the DN is still causing seeks into the checksum file.

(Credit goes to Dhruba for discovering this - filing on his behalf)

Issue Links

• blocks: HBASE-6798 HDFS always read checksum form meta file
• is blocked by: HBASE-6868 Skip checksum is broke; are we double-checksumming by default?