[HDDS-5996] Support getFileChecksum in Ozone

• Type: Bug
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Manager
• Assignee: Wei-Chiu Chuang
• Reporter: Mukul Kumar Singh
• Created: 16/Nov/21 07:14
• Updated: 12/Dec/23 22:47

Issue Links

• is duplicated by: HDDS-2935 Support for getFileChecksum in OzoneFS
• is related to: HDDS-9886 hadoop fs -checksum fails with NoClassDefFoundError: CompositeCrcFileChecksum on Hadoop 2

Sub-Tasks

• Compute MD5MD5CRC file checksum using chunk checksums from DataNodes: Resolved
• Add log4j in the module hadoop-ozone/client: Resolved
• Compute block checksum using chunk checksums: Resolved
• Implement O3FS/OFS getFileChecksum() using file checksum helpers: Resolved
• Compute composite CRC file checksum using chunk checksums from DataNodes: Resolved
• Add file checksum to OmKeyInfo: Resolved
• commitKey() to upload file checksum to OM: Patch Available
• file checksum to support both CRC32 and CRC32C: Resolved
• Refactor BlockInputStream to manage connections using ContainerOperationClient APIs: Open
• Store multiple file checksum at OM: Open
• Add unit tests for OMPBHelper.convert(): Open
• support block composite-crc checksum: Resolved
• distcp cause inconsistent block length between keyInfo and block file length: Open
• getFileChecksum() throws exception in debug mode: Open