[QPID-6535] java.lang.OutOfMemoryError: unable to create new native thread

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.32
• Fix Version/s: qpid-java-6.0
• Component/s: Broker-J
• Assignee: Unassigned
• Reporter: Mark Soderquist
• Created: 08/May/15 13:36
• Updated: 26/Jun/15 02:35
• Resolved: 18/Jun/15 13:56

Description

Getting an OutOfMemoryError after running for several days. Attached is a thread dump showing hundreds of orphaned IoSender threads. At any given time there are fewer than 50 clients but many of them are not "well behaved". Here is the stack trace for the error but I don't really believe the LDAP calls are really the problem:

2015-05-08 03:01:19,000 ERROR [IoReceiver - /10.96.250.6:59923] (server.Main) - Uncaught exception, continuing.
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at com.sun.jndi.ldap.Connection.<init>(Connection.java:234)
at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:136)
at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1608)
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2698)
at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193)
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211)
at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154)
at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)
at javax.naming.InitialContext.init(InitialContext.java:242)
at javax.naming.InitialContext.<init>(InitialContext.java:216)
at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101)
at org.apache.qpid.server.security.auth.manager.SimpleLDAPAuthenticationManagerImpl.createInitialDirContext(SimpleLDAPAuthenticationManagerImpl.java:347)
at org.apache.qpid.server.security.auth.manager.SimpleLDAPAuthenticationManagerImpl.doLDAPNameAuthentication(SimpleLDAPAuthenticationManagerImpl.java:298)
at org.apache.qpid.server.security.auth.manager.SimpleLDAPAuthenticationManagerImpl.access$300(SimpleLDAPAuthenticationManagerImpl.java:72)
at org.apache.qpid.server.security.auth.manager.SimpleLDAPAuthenticationManagerImpl$SimpleLDAPPlainCallbackHandler.handle(SimpleLDAPAuthenticationManagerImpl.java:470)
at org.apache.qpid.server.security.auth.sasl.plain.PlainSaslServer.evaluateResponse(PlainSaslServer.java:83)
at org.apache.qpid.amqp_1_0.transport.ConnectionEndpoint.receiveSaslInit(ConnectionEndpoint.java:908)
at org.apache.qpid.amqp_1_0.type.security.SaslInit.invoke(SaslInit.java:112)
at org.apache.qpid.amqp_1_0.transport.ConnectionEndpoint.receive(ConnectionEndpoint.java:812)
at org.apache.qpid.amqp_1_0.framing.SASLFrameHandler.parse(SASLFrameHandler.java:240)
at org.apache.qpid.server.protocol.v1_0.ProtocolEngine_1_0_0_SASL$4.run(ProtocolEngine_1_0_0_SASL.java:384)
at org.apache.qpid.server.protocol.v1_0.ProtocolEngine_1_0_0_SASL$4.run(ProtocolEngine_1_0_0_SASL.java:380)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at org.apache.qpid.server.protocol.v1_0.ProtocolEngine_1_0_0_SASL.received(ProtocolEngine_1_0_0_SASL.java:379)
at org.apache.qpid.server.protocol.v1_0.ProtocolEngine_1_0_0_SASL.received(ProtocolEngine_1_0_0_SASL.java:66)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:153)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:51)
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:161)
at java.lang.Thread.run(Thread.java:745)

Issue Links

• is related to: QPID-6249 [AMQP 1.0] - JMS AMQP 1.0 Client not sending keep-alive