ActiveMQ
  1. ActiveMQ
  2. AMQ-4312

NIO+SSL Connector fails with SSL exception under high concurrency

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.7.0, 5.8.0
    • Fix Version/s: 5.9.0
    • Component/s: Transport
    • Labels:
    • Environment:

      JDK 1.7.0_09, Windows 7 x64, Linux CentOS5 64-bit

    • Patch Info:
      Patch Available

      Description

      Under high concurrency, the NIO+SSL connector causes client connections to fail with random SSL exceptions (usually bad record MAC or invalid padding) after a period of time.

      For example:

      javax.net.ssl.SSLException: bad record MAC
      	at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
      	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1902)
      	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1855)
      	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:988)
      	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
      	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
      	at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
      	at org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:604)
      	at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:58)
      	at org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:589)
      	at java.io.DataInputStream.readInt(DataInputStream.java:387)
      	at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:275)
      	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
      	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
      	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
      	at java.lang.Thread.run(Thread.java:722)</pre>
      

      I suspect this may be related to AMQ-4135, since we are seeing nearly identical failure modes.

      I am able to duplicate these results fairly consistently using a variant of NIOSSLLoadTest which uses 10 producers, 10 consumers, 10000 messages, and a bytes message of size 4096 with all producers and consumers using separate (multiplexed) sessions.

      1. NIOSSLConcurrencyTest.java
        9 kB
        Craig Condit
      2. AMQ-4312.patch
        0.8 kB
        Craig Condit
      3. AMQ-4312-test.patch
        9 kB
        Craig Condit

        Issue Links

          Activity

          Craig Condit created issue -
          Craig Condit made changes -
          Field Original Value New Value
          Attachment NIOSSLConcurrencyTest.java [ 12569089 ]
          Craig Condit made changes -
          Attachment AMQ-4312.patch [ 12569100 ]
          Craig Condit made changes -
          Patch Info Patch Available [ 10042 ]
          Craig Condit made changes -
          Attachment AMQ-4312-test.patch [ 12569101 ]
          Timothy Bish made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Timothy Bish [ tabish121 ]
          Fix Version/s 5.9.0 [ 12323932 ]
          Resolution Fixed [ 1 ]
          Timothy Bish made changes -
          Link This issue is related to AMQ-4135 [ AMQ-4135 ]

            People

            • Assignee:
              Timothy Bish
              Reporter:
              Craig Condit
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development