[ARTEMIS-1440] Artemis broker.forceFailover not shutting down broker

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.4.0
• Component/s: Broker
• Assignee: Clebert Suconic
• Reporter: Ulf Lilleengen
• Created: 27/Sep/17 11:56
• Updated: 29/Sep/17 13:57
• Resolved: 29/Sep/17 13:57

Description

We have a use case in EnMasse of shutting down the broker using management.

For a time, we have been using forceFailover (https://github.com/apache/activemq-artemis/blob/master/artemis-core-client/src/main/java/org/apache/activemq/artemis/api/core/management/ActiveMQServerControl.java#L1091) with success.

However, this has now stopped working. If I start a vanilla Artemis 2.3.0 instance and attempt to invoke forceFailover through management, I get some errors, and the process never stops after the last message.

13:45:23,825 WARN [org.apache.activemq.artemis.core.server] AMQ222002: Timed out waiting for pool to terminate org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@fcf72cc[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 45]. Interrupting all its threads!
13:45:23,830 WARN [org.apache.activemq.artemis.utils.critical.CriticalAnalyzer] null: java.lang.InterruptedException
at java.lang.Object.wait(Native Method) [rt.jar:1.8.0_66]
at java.lang.Thread.join(Thread.java:1245) [rt.jar:1.8.0_66]
at java.lang.Thread.join(Thread.java:1319) [rt.jar:1.8.0_66]
at org.apache.activemq.artemis.utils.critical.CriticalAnalyzerImpl.stop(CriticalAnalyzerImpl.java:174) [artemis-commons-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.stop(ActiveMQServerImpl.java:1182) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.fail(ActiveMQServerImpl.java:949) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.management.impl.ActiveMQServerControlImpl.forceFailover(ActiveMQServerControlImpl.java:2407) [artemis-server-2.3.0.jar:2.3.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_66]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_66]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]
at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]
at org.apache.activemq.artemis.core.server.management.impl.ManagementServiceImpl.invokeOperation(ManagementServiceImpl.java:747) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.management.impl.ManagementServiceImpl.handleMessage(ManagementServiceImpl.java:391) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.handleManagementMessage(ServerSessionImpl.java:1576) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1363) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1309) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1302) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onSessionSend(ServerSessionPacketHandler.java:690) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:290) [artemis-server-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) [artemis-commons-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase$ExecutorTask.run(ProcessorBase.java:53) [artemis-commons-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.3.0.jar:2.3.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase$ExecutorTask.run(ProcessorBase.java:53) [artemis-commons-2.3.0.jar:2.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

13:45:23,838 INFO [org.apache.activemq.artemis.core.server] AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.3.0 [3bb027ac-a379-11e7-bf7f-eacd4948aa05] stopped, uptime 49.276 seconds

Issue Links

• links to: GitHub Pull Request #1563