[AIRAVATA-2940] Sporadic JPA errors when invoking Registry Server APIs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.17
• Component/s: Registry API
• Environment: staging
• Assignee: Dimuthu
• Reporter: Dimuthu
• Created: 12/Nov/18 18:22
• Updated: 21/Oct/19 20:41

Description

This issue occurs randomly at different registry components. It seems like a general JPA bug or a misuse of JPA APIs in registry code. 

2018-11-10 18:29:28,003 [pool-10-thread-208241] ERROR o.a.a.r.c.a.c.i.ApplicationDeploymentImpl - Error while retrieving application deployment...
org.apache.airavata.registry.cpi.AppCatalogException: <openjpa-2.2.0-r422266:1244990 fatal user error> org.apache.openjpa.persistence.InvalidStateException: The context has been closed. The stack trace at which the context was closed is available if Runtime=TRACE logging is enabled.
at org.apache.airavata.registry.core.app.catalog.resources.LibraryApendPathResource.get(LibraryApendPathResource.java:214)
at org.apache.airavata.registry.core.app.catalog.util.AppCatalogThriftConversion.getApplicationDeploymentDescription(AppCatalogThriftConversion.java:758)
at org.apache.airavata.registry.core.app.catalog.impl.ApplicationDeploymentImpl.getApplicationDeployement(ApplicationDeploymentImpl.java:326)
at org.apache.airavata.registry.api.service.handler.RegistryServerHandler.getApplicationDeployment(RegistryServerHandler.java:1211)
at org.apache.airavata.registry.api.RegistryService$Processor$getApplicationDeployment.getResult(RegistryService.java:14835)
at org.apache.airavata.registry.api.RegistryService$Processor$getApplicationDeployment.getResult(RegistryService.java:14819)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.openjpa.persistence.InvalidStateException: The context has been closed. The stack trace at which the context was closed is available if Runtime=TRACE logging is enabled.
at org.apache.openjpa.kernel.BrokerImpl.assertOpen(BrokerImpl.java:4676)
at org.apache.openjpa.kernel.BrokerImpl.beginOperation(BrokerImpl.java:1930)
at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1503)
at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:933)
at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:570)
at org.apache.airavata.registry.core.app.catalog.resources.LibraryApendPathResource.get(LibraryApendPathResource.java:205)
... 11 common frames omitted
2018-11-10 18:29:28,003 [pool-10-thread-208241] ERROR o.a.a.r.a.s.h.RegistryServerHandler - comet.sdsc.edu_Ultrascan_0091a13a-1fe5-41cf-8708-79a987e3021a
org.apache.airavata.registry.cpi.AppCatalogException: org.apache.airavata.registry.cpi.AppCatalogException: <openjpa-2.2.0-r422266:1244990 fatal user error> org.apache.openjpa.persistence.InvalidStateException: The context has been closed. The stack trace at which the context was closed is available if Runtime=TRACE logging is enabled.
at org.apache.airavata.registry.core.app.catalog.impl.ApplicationDeploymentImpl.getApplicationDeployement(ApplicationDeploymentImpl.java:329)
at org.apache.airavata.registry.api.service.handler.RegistryServerHandler.getApplicationDeployment(RegistryServerHandler.java:1211)
at org.apache.airavata.registry.api.RegistryService$Processor$getApplicationDeployment.getResult(RegistryService.java:14835)
at org.apache.airavata.registry.api.RegistryService$Processor$getApplicationDeployment.getResult(RegistryService.java:14819)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.airavata.registry.cpi.AppCatalogException: <openjpa-2.2.0-r422266:1244990 fatal user error> org.apache.openjpa.persistence.InvalidStateException: The context has been closed. The stack trace at which the context was closed is available if Runtime=TRACE logging is enabled.
at org.apache.airavata.registry.core.app.catalog.resources.LibraryApendPathResource.get(LibraryApendPathResource.java:214)
at org.apache.airavata.registry.core.app.catalog.util.AppCatalogThriftConversion.getApplicationDeploymentDescription(AppCatalogThriftConversion.java:758)
at org.apache.airavata.registry.core.app.catalog.impl.ApplicationDeploymentImpl.getApplicationDeployement(ApplicationDeploymentImpl.java:326)
... 9 common frames omitted
Caused by: org.apache.openjpa.persistence.InvalidStateException: The context has been closed. The stack trace at which the context was closed is available if Runtime=TRACE logging is enabled.
at org.apache.openjpa.kernel.BrokerImpl.assertOpen(BrokerImpl.java:4676)
at org.apache.openjpa.kernel.BrokerImpl.beginOperation(BrokerImpl.java:1930)
at org.apache.openjpa.kernel.BrokerImpl.commit(BrokerImpl.java:1503)
at org.apache.openjpa.kernel.DelegatingBroker.commit(DelegatingBroker.java:933)
at org.apache.openjpa.persistence.EntityManagerImpl.commit(EntityManagerImpl.java:570)
at org.apache.airavata.registry.core.app.catalog.resources.LibraryApendPathResource.get(LibraryApendPathResource.java:205)
... 11 common frames omitted
2018-11-10 18:29:28,144 [pool-30-thread-3] INFO o.a.a.o.s.OrchestratorServerHandler - expId: US3-ADEV_5b40ecc5-7aea-4b42-affc-9a3c6ff7d82c, processId: PROCESS_0fccaaab-7494-4ae7-a1bc-23a0ffd67b8d :- Process status changed event received for status FAILED