Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.2
-
None
-
None
-
CentOS 5.3 4GB RAM 4vCPU
ESX 4
Glassfish 2.2.1, JDK 1.6U18
Eclipselink 1.1.3
400 user load
Description
I have been seeing eclipselink concurrencyexception when the system is under heavy load. Once the exception occurs the entire app hangs. Here is the exception:
[#|2010-02-10T16:15:10.003-0800|WARNING|sun-appserver2.1|org.eclipse.persistence.session.file:/opt/glassfish/domains/domain1/applications/j2ee-modules/webapp/WEB-INF/classes/-BPWebappPu|_ThreadID=16;_ThreadName=httpSSLWorkerThread-8080-0;_RequestID=57c0781d-255f-4f9e-9ccf-41a5c296befe;|
Local Exception Stack:
Exception [EclipseLink-2004] (Eclipse Persistence Services - 1.1.3.v20091002-r5404): org.eclipse.persistence.exceptions.ConcurrencyException
Exception Description: A signal was attempted before wait() on ConcurrencyManager. This normally means that an attempt was made to commit or rollback a transaction before it was started, or to rollback a transaction twice.
at org.eclipse.persistence.exceptions.ConcurrencyException.signalAttemptedBeforeWait(ConcurrencyException.java:84)
at org.eclipse.persistence.internal.helper.ConcurrencyManager.releaseReadLock(ConcurrencyManager.java:478)
at org.eclipse.persistence.internal.identitymaps.CacheKey.releaseReadLock(CacheKey.java:433)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.cloneAndRegisterObject(UnitOfWorkImpl.java:939)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.cloneAndRegisterObject(UnitOfWorkImpl.java:833)
at org.eclipse.persistence.internal.sessions.UnitOfWorkIdentityMapAccessor.getAndCloneCacheKeyFromParent(UnitOfWorkIdentityMapAccessor.java:171)
at org.eclipse.persistence.internal.sessions.UnitOfWorkIdentityMapAccessor.getFromIdentityMap(UnitOfWorkIdentityMapAccessor.java:110)
at org.eclipse.persistence.internal.sessions.IdentityMapAccessor.getFromIdentityMap(IdentityMapAccessor.java:327)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.registerExistingObject(UnitOfWorkImpl.java:3776)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.registerExistingObject(UnitOfWorkImpl.java:3736)
at org.eclipse.persistence.queries.ObjectBuildingQuery.registerIndividualResult(ObjectBuildingQuery.java:362)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:582)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:544)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:485)
at org.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:437)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:569)
at org.eclipse.persistence.queries.ReadAllQuery.registerResultInUnitOfWork(ReadAllQuery.java:904)
at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:490)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:928)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:664)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:889)
at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:458)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:952)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2750)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1181)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1165)
at org.eclipse.persistence.internal.indirection.QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:81)
at org.eclipse.persistence.internal.indirection.QueryBasedValueHolder.instantiate(QueryBasedValueHolder.java:71)
at org.eclipse.persistence.internal.indirection.DatabaseValueHolder.getValue(DatabaseValueHolder.java:83)
at org.eclipse.persistence.internal.indirection.UnitOfWorkValueHolder.instantiateImpl(UnitOfWorkValueHolder.java:161)
at org.eclipse.persistence.internal.indirection.UnitOfWorkValueHolder.instantiate(UnitOfWorkValueHolder.java:230)
at org.eclipse.persistence.internal.indirection.DatabaseValueHolder.getValue(DatabaseValueHolder.java:83)
at org.eclipse.persistence.indirection.IndirectList.buildDelegate(IndirectList.java:220)
at org.eclipse.persistence.indirection.IndirectList.getDelegate(IndirectList.java:386)
at org.eclipse.persistence.indirection.IndirectList$1.<init>(IndirectList.java:514)
at org.eclipse.persistence.indirection.IndirectList.listIterator(IndirectList.java:513)
at org.eclipse.persistence.indirection.IndirectList.iterator(IndirectList.java:477)
at org.apache.olio.webapp.model.ModelFacade.getSocialEvent(ModelFacade.java:583)
at org.apache.olio.webapp.controller.EventAction.process(EventAction.java:71)
at org.apache.olio.webapp.controller.ControllerServlet.process(ControllerServlet.java:61)
at org.apache.olio.webapp.controller.ControllerServlet.doGet(ControllerServlet.java:93)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:427)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:315)
at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:287)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:218)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:98)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:222)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:166)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:648)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:593)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:587)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1093)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:291)
at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:666)
at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:597)
at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:872)
at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:341)
at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:263)
at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:214)
at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:264)
at com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:106)
| #] |
The culprit is the comments. I modified the allocation size of the CommentsRating table to 50000 (it is currently set at 20000) and it appears to fix this.
I have also encountered this problem using Eclipselink 2.0 on Glassfish 2.2.1, I haven't tried Glassfish 3.