Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
qpid-java-broker-7.0.2, qpid-java-broker-7.0.0, qpid-java-broker-7.0.1
-
None
Description
When two concurrent requests are made to delete a group member using REST API for remote replication node, one can successfully remove the node, whilst other can end-up in un-handled MemberNotFoundException which crashes the broker.
An example of remote node delete request
curl -v -k -u admin -X DELETE http://localhost:8080/api/latest/remotereplicationnode/node2/node1
For failed requests an exception like the one below is reported
ERROR [VirtualHostNode-node2-Config] o.a.q.s.u.ServerScopedRuntimeException Exception on node removal from group com.sleepycat.je.rep.MemberNotFoundException: (JE 7.4.5) Node: node1 is not currently a member of the group: BDBHAVirtualHostNodeRestTest.testDeleteMasterNode, it has been removed. at com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:576) at com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:314) at org.apache.qpid.server.store.berkeleydb.replication.ReplicatedEnvironmentFacade.removeNodeFromGroup(ReplicatedEnvironmentFacade.java:1210) at org.apache.qpid.server.virtualhostnode.berkeleydb.BDBHARemoteReplicationNodeImpl.onDelete(BDBHARemoteReplicationNodeImpl.java:139) at org.apache.qpid.server.model.AbstractConfiguredObject$21.onSuccess(AbstractConfiguredObject.java:2759) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:822) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:664) at org.apache.qpid.server.model.AbstractConfiguredObject$ChainedSettableFuture.set(AbstractConfiguredObject.java:2710) at org.apache.qpid.server.model.AbstractConfiguredObject$21$1.onSuccess(AbstractConfiguredObject.java:2765) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:400) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:183) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911) at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:645) at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.addListener(AbstractFuture.java:101) at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1209) at org.apache.qpid.server.model.AbstractConfiguredObject.addFutureCallback(AbstractConfiguredObject.java:2884) at org.apache.qpid.server.model.AbstractConfiguredObject$21.onSuccess(AbstractConfiguredObject.java:2760) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2894) at org.apache.qpid.server.model.AbstractConfiguredObject$24$1.run(AbstractConfiguredObject.java:2890) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.qpid.server.model.AbstractConfiguredObject$24.onSuccess(AbstractConfiguredObject.java:2889) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1237) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor$1$1.run(TaskExecutorImpl.java:415) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor$1$1.run(TaskExecutorImpl.java:411) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor$1.run(TaskExecutorImpl.java:410) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.qpid.server.bytebuffer.QpidByteBufferFactory.lambda$null$0(QpidByteBufferFactory.java:464) at java.lang.Thread.run(Thread.java:748)