[HADOOP-7443] Add CRC32C as another DataChecksum implementation

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.23.0
• Component/s: io, util
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 06/Jul/11 04:35
• Updated: 14/May/16 11:59
• Resolved: 13/Jul/11 19:42

Description

CRC32C is another checksum very similar to our existing CRC32, but with a different polynomial. The chief advantage of this other polynomial is that SSE4.2 includes hardware support for its calculation. HDFS-2080 is the umbrella JIRA which proposes using this new polynomial to save substantial amounts of CPU.

Issue Links

• is blocked by: HADOOP-7444 Add Checksum API to verify and calculate checksums "in bulk"
• is part of: HDFS-2080 Speed up DFS read path by lessening checksum overhead
• relates to: HADOOP-13147 Constructors must not call overrideable methods