[FLINK-14956] MemoryMappedBoundedData Compressed Buffer Slicer

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Component/s: Runtime / Network
• Assignee: Unassigned
• Reporter: Nicholas Jiang
• Created: 27/Nov/19 01:35
• Updated: 29/Nov/19 07:24
• Resolved: 29/Nov/19 07:24

Description

MemoryMappedBoundedData, implementation of BoundedData simply through ByteBuffers backed by memory, uses CompressedBufferSlicer which is implementation of BoundedData.Reader to slice next buffer with uncompress. CompressedBufferSlicer reads BoundedData by LZ4SafeDecompressor decompressing byte buffer.When FileChannelMemoryMappedBoundedData tries to write buffer, this uses LZ4Compressor to compress buffer to improve I/O performance.

Compress read process：

Compress write process：

CPU/IO performance comparsion chart：