Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
2.0.1
-
None
-
None
Description
When removing a service (via the API, for instance curl -u $ambariuser -i -H 'X-Requested-By: ambari' -X DELETE http://ambari-server:8080/api/v1/clusters/CLUSTER01/services/AMBARI_METRICS), the entries in alert_definition table are not removed.
When re-adding the service, the uni_alert_def_name constraint fails, leading to a deadlock (looks like the lock is not released properly).
Constraint exception:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uni_alert_def_name" Detail: Key (cluster_id, definition_name)=(2, metrics_monitor_process_percent) already exists. Error Code: 0 Call: INSERT INTO alert_definition (definition_id, cluster_id, component_name, definition_name, description, ENABLED, HASH, ignore_host, label, schedule_interval, scope, service_name, alert_source, source_type) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) bind => [14 parameters bound] at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1611) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:962) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:631) at org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatch(ParameterizedSQLBatchWritingMechanism.java:149) at org.eclipse.persistence.internal.databaseaccess.ParameterizedSQLBatchWritingMechanism.executeBatchedStatements(ParameterizedSQLBatchWritingMechanism.java:134) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.writesCompleted(DatabaseAccessor.java:1836) at org.eclipse.persistence.internal.sessions.AbstractSession.writesCompleted(AbstractSession.java:4244) at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.writesCompleted(UnitOfWorkImpl.java:5594) at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges(RepeatableWriteUnitOfWork.java:453) at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:863) at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:963) at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:207) at org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult(QueryImpl.java:517) at org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:400) at org.apache.ambari.server.orm.dao.DaoUtils.selectSingle(DaoUtils.java:71) at org.apache.ambari.server.orm.dao.AlertDispatchDAO.findDefaultServiceGroup(AlertDispatchDAO.java:346) at org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:53) at org.apache.ambari.server.orm.dao.AlertDefinitionDAO.create(AlertDefinitionDAO.java:307) at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:68) at org.apache.ambari.server.events.listeners.alerts.AlertServiceStateListener.onAmbariEvent(AlertServiceStateListener.java:148) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74) at com.google.common.eventbus.EventBus.dispatch(EventBus.java:314) at com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34) at com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "uni_alert_def_name" Detail: Key (cluster_id, definition_name)=(2, metrics_monitor_process_percent) already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
Stacktrace
"qtp-client-24" prio=10 tid=0x00007fdc656e4800 nid=0xe79 waiting on condition [0x00007fdc266bb000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000079e3364f0> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945) at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persistConfigMapping(ConfigGroupImpl.java:419) at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.saveIfPersisted(ConfigGroupImpl.java:456) at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persist(ConfigGroupImpl.java:323) at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider.updateConfigGroups(ConfigGroupResourceProvider.java:600) - locked <0x00000007a0067b30> (a org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider) at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider.access$300(ConfigGroupResourceProvider.java:58) at org.apache.ambari.server.controller.internal.ConfigGroupResourceProvider$4.invoke(ConfigGroupResourceProvider.java:269)