[HDFS-14417] CryptoInputStream decrypt methods can avoid usage of intermediate buffers

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: hdfs-client, performance
• Assignee: Unassigned
• Reporter: Sahil Takiar
• Created: 05/Apr/19 23:26
• Updated: 09/Apr/19 18:52

Description

Filing this as a follow up to this review comment on HDFS-3246. In CryptoInputStream the decrypt methods rely on temporary buffers to help decrypt the given encrypted data. The decrypt methods work by copying the input data chunk by chunk into an "input" buffer and then passing the "input" buffer and an "output" buffer to a decryption method that reads from the "input" buffer and writes to the "output" buffer. The contents of the "output" buffer are then copied back into the user buffer. Then the method moves onto the next chunk of the user buffer.

Instead of copying all the data between these buffers, we should be able to decrypt them in-place - e.g. the input and output buffer are the same. As the comment points out, OpenSSL supports this.

At the very least, we should be able to remove the usage of the "output" buffer and just pass the user buffer directly to the decryption classes.