[CASSANDRA-4297] Use java NIO as much as possible when streaming compressed SSTables

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 1.2.0 beta 1
• Labels: streaming
• Assignee: Yuki Morishita Assign to me
• Reporter: Yuki Morishita

Authors

Yuki Morishita

Reviewers

Jonathan Ellis

• Created: 30/May/12 17:50
• Updated: 16/Apr/19 09:32
• Resolved: 13/Jun/12 17:28

Description

Back in 0.8, streaming uses java NIO (FileChannel#transferTo/transferFrom) to perform zero copy file transfer between nodes. Since 1.0, in order to add new features like sstable compression and internode encryption we had to switch to java IO Input/OutputStreams. What we currently do to transfer compressed SSTable is, in source node, 1) decompress chunk in SSTable, 2) compress using LZF for network, and in destination node, 3) decompress using LZF as reading from socket, 4) compress for SSTable on disk.

Now, 1.1 comes out with SSTable compression turned on by default. It is reasonable to transfer compressed file as is using NIO instead of decompress/compress in source node.