Affects Version/s: None
Fix Version/s: 2.6.0
Release Note:HDFS now supports the option to configure AES encryption for block data transfer. AES offers improved cryptographic strength and performance over the prior options of 3DES and RC4.
HDFS-3637, Aaron T. Myers added support for encrypting the DataTransferProtocol, it was a great work.
It utilizes SASL Digest-MD5 mechanism (use Qop: auth-conf), it supports three security strength:
- high 3des or rc4 (128bits)
- medium des or rc4(56bits)
- low rc4(40bits)
3des and rc4 are slow, only tens of MB/s,
I will give more detailed performance data in future. Absolutely it’s bottleneck and will vastly affect the end to end performance.
AES(Advanced Encryption Standard) is recommended as a replacement of DES, it’s more secure; with AES-NI support, the throughput can reach nearly 2GB/s, it won’t be the bottleneck any more, AES and CryptoCodec work is supported in
HADOOP-10150, HADOOP-10603 and HADOOP-10693 (We may need to add a new mode support for AES).
This JIRA will use AES with AES-NI support as encryption algorithm for DataTransferProtocol.