[KAFKA-13799] Improve documentation for Kafka zero-copy

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Component/s: documentation
• Assignee: RivenSun
• Reporter: RivenSun
• Created: 05/Apr/22 11:04
• Updated: 27/Apr/22 03:54
• Resolved: 20/Apr/22 17:32

Description

Via documentation https://kafka.apache.org/documentation/#maximizingefficiency
and https://kafka.apache.org/documentation/#networklayer ,
We can know that Kafka combines pagecache and zero-copy when reading messages in files on disk, which greatly improves the consumption rate of messages.
But after browsing the source code:
Look directly at the FileRecords.writeTo(...) method,
1. Only PlaintextTransportLayer.transferFrom() uses fileChannel.transferTo(), and the bottom layer calls the sendfile method to implement zero-copy data transfer.
2. The logic of the SslTransportLayer.transferFrom() method: 

fileChannel.read(fileChannelBuffer, pos) 
-> 
sslEngine.wrap(src, netWriteBuffer) 
-> 
flush(ByteBuffer buf) && socketChannel.write(buf)

That is, first read the data on the disk or directly from the page cache, then encrypt the data, and finally send the encrypted data to the network. FileChannel.transferTo() is not used in the whole process.

 

Conclusion: 

PlaintextTransportLayer and SslTransportLayer both use pagecache, but SslTransportLayer does not implement zero-copy.

Issue Links

• links to: GitHub Pull Request #12052