[AMQ-3819] high cpu with stomp+nio+ssl and many subscriptions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.6.0
• Fix Version/s: 5.6.0
• Component/s: STOMP

Environment

CentOS 6, RC of 5.6.0

java version "1.6.0_22"
OpenJDK Runtime Environment (IcedTea6 1.10.6) (rhel-1.43.1.10.6.el6_2-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)

• Assignee: Timothy A. Bish
• Reporter: R.I.Pienaar
• Created: 25/Apr/12 13:53
• Updated: 01/May/12 13:29
• Resolved: 01/May/12 13:29

Description

Switching an existing workload from a transport:

<transportConnector name="verified_stompssl" uri="stomp+ssl://0.0.0.0:6165?needClientAuth=true"/>

to

<transportConnector name="verified_stompssl" uri="stomp+nio+ssl://0.0.0.0:6165?needClientAuth=true"/>

showed the CPU profile to go from 1-5% to 300% constantly on a 8 core server

I was able to recreate this using a ruby client @ http://devco.net/rip/amq_560_stomp_nio_ssl_tester.rb

The important combinations are:

• I am connecting to a stomp+nio+ssl port
• I am creating the subscriptions to the 10 queus and topics

If I change either of these variables - like just commenting out the loop that does those subscriptions - then the CPU load is acceptable.

I analysed the running VM with VisualVM and found that transport.nio.NIOSSLTransport.serviceRead() is the busy thread.

My activemq.xml is:

<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="amq1" useJmx="true" persistent="true" schedulePeriodForDestinationPurge="60000">
        <destinationPolicy>
          <policyMap>
            <policyEntries>
              <policyEntry topic=">" producerFlowControl="false"/>
              <policyEntry queue="*.reply.>" gcInactiveDestinations="true" inactiveTimoutBeforeGC="120000" />
            </policyEntries>
          </policyMap>
        </destinationPolicy>

        <managementContext>
          <managementContext connectorPort="1099" jmxDomainName="org.apache.activemq"/>
        </managementContext>

        <plugins>
          <statisticsBrokerPlugin/>
          <simpleAuthenticationPlugin>
            <users>
              <authenticationUser username="test" password="test" groups="admins,everyone"/>
            </users>
          </simpleAuthenticationPlugin>
          <authorizationPlugin>
            <map>
              <authorizationMap>
                <authorizationEntries>
                  <authorizationEntry queue=">" write="admins" read="admins" admin="admins" />
                  <authorizationEntry topic=">" write="admins" read="admins" admin="admins" />
                </authorizationEntries>
              </authorizationMap>
            </map>
          </authorizationPlugin>
        </plugins>

        <sslContext>
           <sslContext
                keyStore="keystore.jks" keyStorePassword="ohshahCu"
                trustStore="truststore.jks" trustStorePassword="ohshahCu"
           />
        </sslContext>

        <systemUsage>
          <systemUsage>
            <memoryUsage>
              <memoryUsage limit="200 mb" />
            </memoryUsage>
            <storeUsage>
              <storeUsage limit="1 gb" />
            </storeUsage>
            <tempUsage>
              <tempUsage limit="1 gb" />
            </tempUsage>
          </systemUsage>
        </systemUsage>

        <transportConnectors>
          <transportConnector name="openwire"  uri="tcp://0.0.0.0:6166"/>
          <transportConnector name="stomp+nio" uri="stomp+nio://0.0.0.0:6163"/>
          <transportConnector name="stompssl"  uri="stomp+ssl://0.0.0.0:6164"/>
          <transportConnector name="verified_stompssl"  uri="stomp+nio+ssl://0.0.0.0:6165?needClientAuth=true"/>
        </transportConnectors>
    </broker>

    <import resource="jetty.xml"/>
</beans>