[HDDS-11445] HBase support Phase II

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0 (obsolete, don't use)
• Assignee: Unassigned
• Reporter: Wei-Chiu Chuang
• Created: 10/Sep/24 18:13
• Updated: 10/Oct/24 16:55

Description

In continuation of HDDS-7593. With HDDS-7593 and to be released in the upcoming Ozone 2.0, HBase can run on Ozone with good performance.

This jira tracks follow-up bug fixes and improvements.

Issue Links

• is related to: HDDS-11521 Race condition between pipeline close and block allocation causes client aborts
• is related to: HDDS-11449 Ambigious "Ignoring ozone.fs.hsync.enabled because HBase enhancements are disallowed" logs
• is related to: HDDS-11558 Make OM client retry idempotent
• relates to: HDDS-7593 Supporting HSync and lease recovery

Sub-Tasks

• Support incremental ChunkBuffer checksum calculation: Open
• Freon DN Echo should skip writing payload to ratis log: In Progress
• Add metrics for OzoneChecksumException: Open
• Explore client retry optimizations after write() and hsync() are desynced: Open
• Add test case to cover HDDS-9930 batch key deletion: Open
• [hsync] Update OpenTelemetry traces in the write path: In Progress
• [hsync] Improve test coverage for LeaseRecoveryClientDNHandler: Open
• [Discuss] Colocate Ozone client and pipeline leader to reduce latency: Open
• [hsync] Optimize FilePerBlockStrategy.writeChunk(): Open
• Manage Netty native memory consumption: Open
• Any open key rename should ideally be blocked: Open
• [hsync] Adopt RATIS-1994 to reduce hsync latency: Patch Available
• [hsync] Freon DN Echo to support --wait-for-commit parameter: Open
• Make key visible immediately upon create(): Open
• Add hsync and lease recovery Documentation: In Progress
• Increase DataNode XceiverServerGrpc event loop group size: Open
• New tool to micro benchmark hbase performance in Ozone: Open