[HDDS-10283] Apply zero copy to Ozone client reads chunks

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Client
• Labels: performance
• Assignee: Unassigned
• Reporter: Duong
• Created: 02/Feb/24 19:34
• Updated: 04/Feb/24 01:34

Description

Ozone client, or ChunkInputStream, reads chunks via grpc call. The chunk data is included in the `ReadChunkResponseProto` serialized by GRPC. By default, GRPC serializes bytes data by copying the data it receives from network (in the native memory zone) to heap buffers (LiteralByteString). The ChunkInputStream uses these buffers to back the read.

We can apply zero-copy marshaller (ZeroCopyMessageMarshaller) to client reads to instruct GRPC to serialize chunks by wrapping the direct buffers of data received from the network. This will eliminate the need to allocate extra heap buffers when reading data.

 

 

Issue Links

• is blocked by: HDDS-10288 Checksum to support direct buffers
• relates to: HDDS-9668 Zero-Copy in EC GRPC (write)
• relates to: HDDS-9843 Ozone client high memory (heap) utilization