Description
The use of ByteArrayOutputStream causes unexpected memory copies that causes unintuitive memory usage for certain combinations of the configuration parameters. Replacing it by more fine grained memory buffer management would fix this, reduce memory usage and eliminate a memory copy.
Attachments
Issue Links
- is superceded by
-
HADOOP-13560 S3ABlockOutputStream to support huge (many GB) file writes
-
- Resolved
-