[HDFS-3177] Allow DFSClient to find out and use the CRC type being used for a file.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.3, 2.0.2-alpha
• Component/s: datanode, hdfs-client
• Hadoop Flags: Reviewed
• Assignee: Kihwal Lee
• Reporter: Kihwal Lee
• Created: 02/Apr/12 16:43
• Updated: 11/Oct/12 17:46
• Resolved: 24/Aug/12 14:27

Description

To support HADOOP-8060, DFSClient should be able to find out the checksum type being used for files in hdfs.
In my prototype, DataTransferProtocol was extended to include the checksum type in the blockChecksum() response. DFSClient uses it in getFileChecksum() to determin the checksum type. Also append() can be configured to use the existing checksum type instead of the configured one.

Issue Links

• is blocked by: HADOOP-8239 Extend MD5MD5CRC32FileChecksum to show the actual checksum type being used
• is blocked by: HADOOP-8240 Allow users to specify a checksum type on create()
• is part of: HADOOP-8060 Add a capability to discover and set checksum types per file.