-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.5.0
-
Component/s: Ozone Client
-
Labels:
In HDDS-2331, we found that Ozone client allocates a ByteBuffer with chunk size (e.g. 16MB ) to store data, unregarded the actual data size. The ByteBuffer will create a byte[] with chunk size. When the ByteBuffer is wrapped to a ByteString the byte[] remains in the ByteString.
As a result, when the actual data size is small (e.g. 1MB), a lot of memory spaces (15MB) are wasted.
In this JIRA, we refactor BlockOutputStream so that the buffering becomes more flexible. In a later JIRA (HDDS-2386), we implement chunk buffer using a list of smaller buffers which are allocated only if needed.
