Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Won't Do
-
None
-
None
-
None
Description
Ratis streaming supports writeAsync(File, position, count) API, which is backed by Netty FileRegion and FileChannel.transferTo.
Let's introduce a new interface FileRegionStreamOutput to implement zero copy in OzoneDataStreamOutput, KeyDataStreamOutput, BlockDataStreamOutputEntry and BlockDataStreamOutput.
Our tests shows Streaming(FileRegion) API have better performance over Async API.
And it's a good performance reference for the Streaming(ByteBuffer) API.
Test A (single 10GB key)
API | Min | Max | Avg |
---|---|---|---|
Async | 64s | 72s | 68s |
Streaming (MappedByteBuffer) | 55s | 68s | 62s |
Streaming (FileRegion) | 55s | 64s | 61s |
Test B (24x 1GB keys from 1 client)
API | Min | Max | Avg |
---|---|---|---|
Async | 48s | 54s | 51s |
Streaming (MappedByteBuffer) | 32s | >630s | 33s |
Streaming (FileRegion) | 36s | 39s | 38s |