[AMQ-5086] vm transport create=false&waitForStart race condition

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.7.0, 5.8.0, 5.9.0
• Fix Version/s: 5.10.1, 5.11.0
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Christian Mamen
• Created: 06/Mar/14 18:30
• Updated: 16/Dec/14 00:00
• Resolved: 09/Jun/14 19:46

Description

Experience this bug on 5.7.0, I think this is the same on the trunk

using vm transport for a client to connect to an embedded broker, in a multithreaded application, I'm experiencing a an error (sometimes) which appears to be a race condition at startup.

Im using create=false and waitForStart to create a connectionFactory for a client connection
vm://ApplicationName?create=false&waitForStart=120000

The broker service is started in a seperate thread

the client connection is started first. but surprisingly it tries start the brokers transport connector. An apparent glitch follows when the broker service stops and re-start the transport.

2014-03-05 11:07:57,626 [ClientConnection_thread] INFO  org.apache.activemq.broker.TransportConnector - Connector vm://ApplicationName Started
[...]
2014-03-05 11:08:07,009 [Main_thread] INFO  org.apache.activemq.broker.TransportConnector - Connector vm://ApplicationName Stopped
2014-03-05 11:08:07,011 [Main_thread] INFO  org.apache.activemq.broker.TransportConnector - Connector vm://ApplicationName Started

I look into the activemq source and saw this:

BrokerService.class

public void start() throws Exception {
[...]
    // in jvm master slave, lets not publish over existing broker till we get the lock
    final BrokerRegistry brokerRegistry = BrokerRegistry.getInstance();
    if (brokerRegistry.lookup(getBrokerName()) == null) {
            brokerRegistry.bind(getBrokerName(), BrokerService.this);
    }
    startPersistenceAdapter(startAsync);
    startBroker(startAsync);
    brokerRegistry.bind(getBrokerName(), BrokerService.this);

VMTransportFactory.class

    private BrokerService lookupBroker(final BrokerRegistry registry, final String brokerName, int waitForStart) {
        BrokerService broker = null;
        synchronized(registry.getRegistryMutext()) {
            broker = registry.lookup(brokerName);
            if (broker == null && waitForStart > 0) {
                final long expiry = System.currentTimeMillis() + waitForStart;
                while (broker == null  && expiry > System.currentTimeMillis()) {
                    long timeout = Math.max(0, expiry - System.currentTimeMillis());
                    try {
                        LOG.debug("waiting for broker named: " + brokerName + " to start");
                        registry.getRegistryMutext().wait(timeout);
                    } catch (InterruptedException ignored) {
                    }
                    broker = registry.lookup(brokerName);
                }
            }
        }
        return broker;
    }

It appears that create=false and waitForStart only waits for the broker to be added to the BrokerRegistry. However when the brokerService is starts, it seems that the broker is added to the registry before it is started.

I believe some synchronization is missing make the VMTransportFactory wait for the broker not only to be added to the registry, but also fully started.