Details

    • Type: Improvement Improvement
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 5.2.0
    • Fix Version/s: 5.3.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently we don't allow restarting the stopped broker and instead developers need to create another instance of the broker and start it. Usually this is a desired behavior and should remain a default one. But we should provide an option to people wanting to start stopped brokers. The proposed enhancement will introduce

      public void start(boolean force);
      

      method, which will force the broker to start.

        Activity

        Hide
        Dejan Bosanac added a comment -

        Fixed in SVN revision 773569.

        Broker's context is not entirely cleaned on stop. I tried to find most of those places and cover them with a test case. If you experience a problem in behavior of the restarted broker, please reopen this issue with your configuration (and use case)

        Show
        Dejan Bosanac added a comment - Fixed in SVN revision 773569. Broker's context is not entirely cleaned on stop. I tried to find most of those places and cover them with a test case. If you experience a problem in behavior of the restarted broker, please reopen this issue with your configuration (and use case)
        Hide
        Raphaël Ramirez added a comment -

        I get a BrokerStoppedException when I try to restart a stopped Broker.

        Show
        Raphaël Ramirez added a comment - I get a BrokerStoppedException when I try to restart a stopped Broker.
        Hide
        Raphaël Ramirez added a comment -

        I would like to do the following with an Master-slave with database lock configuration :

        • if a database problem occurs (loss of connection, etc.)
        • the master looses the database lock : "No longer able to keep the exclusive lock so giving up being a master" and a slave become the new master as usual
        • but instead of shutting the broker down completely, I'm trying to make it become a slave (able to become a master again)

        I use the attached ActiveMQThreadBrokerFactory in Spring as following (also see AMQ-1273) :

        SpringContext.xml
        	<bean id="brokerFactory" class="app.messaging.ActiveMQThreadBrokerFactory">
        		<property name="brokerService" ref="brokerService"/><!-- start must be false on this broker -->
        		<property name="exceptionHandler" ref="exceptionHandler"/>
        	</bean>
        
          <amq:broker useJmx="true" persistent="true" brokerName="broker2" id="brokerService" start="false">
            <amq:managementContext>
            	<amq:managementContext useMBeanServer="true" createMBeanServer="false" createConnector="false"/> <!-- use the JVM MBean server -->
            </amq:managementContext>
         	<amq:persistenceAdapter>
            	<amq:jdbcPersistenceAdapter dataSource="#persistenceDataSource" createTablesOnStartup="true" useDatabaseLock="true" />
        	</amq:persistenceAdapter>
        	<amq:transportConnectors>
        		<amq:transportConnector uri="tcp://localhost:50000" />
        		<amq:transportConnector name="stomp" uri="stomp://localhost:50001" />
        	</amq:transportConnectors>
          </amq:broker>
        
        	<bean id="persistenceDataSource" class="...." init-method="init" destroy-method="close">
        		....
        	</bean>
        
        	<!-- modified exception handler to avoid logging "Lock wait timeout exceeded" messages (normal slave behaviour) -->
        	<bean id="exceptionHandler" class="app.messaging.LockExceptionHandler"/>
        


        With this configuration, the master->slave behavior works (looping to acquire the lock). But when the lock is obtained again, it happens that the broker is actually replaced by an ErrorBroker (replaced during the previous stop):

        BrokerService.java
            protected Broker createBroker() throws Exception {
        ...
                    public void stop() throws Exception {
                        old = this.next.getAndSet(new ErrorBroker("Broker has been stopped: " + this) {
                            // Just ignore additional stop actions.
                            public void stop() throws Exception {
                            }
                        });
                        old.stop();
                    }
        ...
        

        So I get the following stacktrace when nowMasterBroker is called :

        INFO org.apache.activemq.store.jdbc.DefaultDatabaseLocker | Becoming the master on dataSource: app.db.ConfigurableDbcpDataSource@deeebd
        ERROR org.apache.activemq.broker.BrokerService | Failed to start ActiveMQ JMS Message Broker. Reason: org.apache.activemq.broker.BrokerStoppedException: Broker has been stopped: org.apache.activemq.broker.BrokerService$3@1f489d9
        org.apache.activemq.broker.BrokerStoppedException: Broker has been stopped: org.apache.activemq.broker.BrokerService$3@1f489d9
        at org.apache.activemq.broker.ErrorBroker.nowMasterBroker(ErrorBroker.java:297)
        at org.apache.activemq.broker.MutableBrokerFilter.nowMasterBroker(MutableBrokerFilter.java:307)
        at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.start(JDBCPersistenceAdapter.java:208)
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:449)
        at org.apache.activemq.broker.BrokerService.start(BrokerService.java:420)
        at app.messaging.ActiveMQThreadBrokerFactory$BrokerThread.run(ActiveMQThreadBrokerFactory.java:64)
        INFO org.apache.activemq.broker.BrokerService | ActiveMQ Message Broker (broker2, ID:PC030-52124-1958729374192-0:0) is shutting down
        INFO org.apache.activemq.broker.TransportConnector | Connector tcp://localhost:50001 Stopped
        INFO org.apache.activemq.broker.TransportConnector | Connector stomp Stopped
        INFO org.apache.activemq.broker.BrokerService | ActiveMQ JMS Message Broker (broker2, ID:PC030-52124-1958729374192-0:0) stopped
        ERROR app.messaging.ActiveMQThreadBrokerFactory | Init brokerService error. Will try again in 10000 ms

        I assume that checking forceStart in BrokerService.getBroker() may help by calling again createBroker() for an existing (non null) broker.

        (Another thing is that with my configuration, setting lockAcquireSleepInterval in <amq:jdbcPersistenceAdapter/> in the Spring context has no effect (the default value is always used). This is why I have to set it (twice) in ActiveMQThreadBrokerFactory.java.)

        Show
        Raphaël Ramirez added a comment - I would like to do the following with an Master-slave with database lock configuration : if a database problem occurs (loss of connection, etc.) the master looses the database lock : "No longer able to keep the exclusive lock so giving up being a master" and a slave become the new master as usual but instead of shutting the broker down completely, I'm trying to make it become a slave (able to become a master again) I use the attached ActiveMQThreadBrokerFactory in Spring as following (also see AMQ-1273 ) : SpringContext.xml <bean id= "brokerFactory" class= "app.messaging.ActiveMQThreadBrokerFactory" > <property name= "brokerService" ref= "brokerService" /> <!-- start must be false on this broker --> <property name= "exceptionHandler" ref= "exceptionHandler" /> </bean> <amq:broker useJmx= "true" persistent= "true" brokerName= "broker2" id= "brokerService" start= "false" > <amq:managementContext> <amq:managementContext useMBeanServer= "true" createMBeanServer= "false" createConnector= "false" /> <!-- use the JVM MBean server --> </amq:managementContext> <amq:persistenceAdapter> <amq:jdbcPersistenceAdapter dataSource= "#persistenceDataSource" createTablesOnStartup= "true" useDatabaseLock= "true" /> </amq:persistenceAdapter> <amq:transportConnectors> <amq:transportConnector uri= "tcp://localhost:50000" /> <amq:transportConnector name= "stomp" uri= "stomp://localhost:50001" /> </amq:transportConnectors> </amq:broker> <bean id= "persistenceDataSource" class= "...." init-method= "init" destroy-method= "close" > .... </bean> <!-- modified exception handler to avoid logging "Lock wait timeout exceeded" messages (normal slave behaviour) --> <bean id= "exceptionHandler" class= "app.messaging.LockExceptionHandler" /> With this configuration, the master->slave behavior works (looping to acquire the lock). But when the lock is obtained again, it happens that the broker is actually replaced by an ErrorBroker (replaced during the previous stop): BrokerService.java protected Broker createBroker() throws Exception { ... public void stop() throws Exception { old = this .next.getAndSet( new ErrorBroker( "Broker has been stopped: " + this ) { // Just ignore additional stop actions. public void stop() throws Exception { } }); old.stop(); } ... So I get the following stacktrace when nowMasterBroker is called : INFO org.apache.activemq.store.jdbc.DefaultDatabaseLocker | Becoming the master on dataSource: app.db.ConfigurableDbcpDataSource@deeebd ERROR org.apache.activemq.broker.BrokerService | Failed to start ActiveMQ JMS Message Broker. Reason: org.apache.activemq.broker.BrokerStoppedException: Broker has been stopped: org.apache.activemq.broker.BrokerService$3@1f489d9 org.apache.activemq.broker.BrokerStoppedException: Broker has been stopped: org.apache.activemq.broker.BrokerService$3@1f489d9 at org.apache.activemq.broker.ErrorBroker.nowMasterBroker(ErrorBroker.java:297) at org.apache.activemq.broker.MutableBrokerFilter.nowMasterBroker(MutableBrokerFilter.java:307) at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.start(JDBCPersistenceAdapter.java:208) at org.apache.activemq.broker.BrokerService.start(BrokerService.java:449) at org.apache.activemq.broker.BrokerService.start(BrokerService.java:420) at app.messaging.ActiveMQThreadBrokerFactory$BrokerThread.run(ActiveMQThreadBrokerFactory.java:64) INFO org.apache.activemq.broker.BrokerService | ActiveMQ Message Broker (broker2, ID:PC030-52124-1958729374192-0:0) is shutting down INFO org.apache.activemq.broker.TransportConnector | Connector tcp://localhost:50001 Stopped INFO org.apache.activemq.broker.TransportConnector | Connector stomp Stopped INFO org.apache.activemq.broker.BrokerService | ActiveMQ JMS Message Broker (broker2, ID:PC030-52124-1958729374192-0:0) stopped ERROR app.messaging.ActiveMQThreadBrokerFactory | Init brokerService error. Will try again in 10000 ms I assume that checking forceStart in BrokerService.getBroker() may help by calling again createBroker() for an existing (non null) broker. (Another thing is that with my configuration, setting lockAcquireSleepInterval in <amq:jdbcPersistenceAdapter/> in the Spring context has no effect (the default value is always used). This is why I have to set it (twice) in ActiveMQThreadBrokerFactory.java .)
        Hide
        Scott Clasen added a comment -

        I see the same stack trace today in 5.3.1 after trying to force start a BrokerService that was stopped. Is there a way to replace the ErrorBroker?

        Show
        Scott Clasen added a comment - I see the same stack trace today in 5.3.1 after trying to force start a BrokerService that was stopped. Is there a way to replace the ErrorBroker?

          People

          • Assignee:
            Dejan Bosanac
            Reporter:
            Dejan Bosanac
          • Votes:
            2 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development