Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2561

Optionally support OpenSSL for SSL/TLS

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None

      Description

      JDK's `SSLEngine` is unfortunately a bit slow (KAFKA-2431 covers this in more detail). We should consider supporting OpenSSL for SSL/TLS. Initial experiments on my laptop show that it performs a lot better:

      start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, config
      2015-09-21 14:41:58:245, 2015-09-21 14:47:02:583, 28610.2295, 94.0081, 30000000, 98574.6111, Java 8u60/server auth JDK SSLEngine/TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
      2015-09-21 14:38:24:526, 2015-09-21 14:40:19:941, 28610.2295, 247.8900, 30000000, 259931.5514, Java 8u60/server auth OpenSslEngine/TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
      2015-09-21 14:49:03:062, 2015-09-21 14:50:27:764, 28610.2295, 337.7751, 30000000, 354182.9000, Java 8u60/plaintext
      

      Extracting the throughput figures:

      • JDK SSLEngine: 94 MB/s
      • OpenSSL SSLEngine: 247 MB/s
      • Plaintext: 337 MB/s (code from trunk, so no zero-copy due to KAFKA-2517)

      In order to get these figures, I used Netty's `OpenSslEngine` by hacking `SSLFactory` to use Netty's `SslContextBuilder` and made a few changes to `SSLTransportLayer` in order to workaround differences in behaviour between `OpenSslEngine` and JDK's SSLEngine (filed https://github.com/netty/netty/issues/4235 and https://github.com/netty/netty/issues/4238 upstream).

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ijuma Ismael Juma
            • Votes:
              1 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated: