Per Todd Lipcon's comment in
Whenever a native decompression codec is being used, ... we generally have the following copies:
1) Socket -> DirectByteBuffer (in SocketChannel implementation)
2) DirectByteBuffer -> byte (in SocketInputStream)
3) byte -> Native buffer (set up for decompression)
4*) decompression to a different native buffer (not really a copy - decompression necessarily rewrites)
5) native buffer -> byte
with the proposed improvement we can hopefully eliminate #2,#3 for all applications, and #2,#3,and #5 for libhdfs.
The interfaces in the attached patch attempt to address:
A - Compression and decompression based on ByteBuffers (
B - Zero-copy compression and decompression (HDFS-3051)
C - Provide the caller a way to know how the max space required to hold compressed output.