Redelivery on a failure doesn't respect the initialRedeliveryDelay on the first attempt.

• Key: AMQ-1847
• Type: Bug
• Status: Reopened
• Priority: Major
• Assignee: Unassigned
• Reporter: Veit Guna
• Component/s: Broker
• Affects Version/s: 5.1.0
• Fix Version/s: 5.3.0

Description

Hi.

I'm using ActiveMQ 5.1 together with Spring's DefaultMessageListenerContainer to enable MDPs. I'm using the redeliveryPolicy to enable redelivery on failures during consumption of a message. ActiveMQ configuration looks like this:

<camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring">

<route errorHandlerRef="deadLetterErrorHandler">
<from uri="activemq:NEW" />
</route>
</camelContext>

<bean id="deadLetterErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder">
<property name="redeliveryPolicy" ref="redeliveryPolicyConfig"/>
</bean>

<bean id="redeliveryPolicyConfig" class="org.apache.camel.processor.RedeliveryPolicy">
<property name="maximumRedeliveries" value="4"/>
<property name="initialRedeliveryDelay" value="30000"/>
<property name="useExponentialBackOff" value="true"/>
<property name="backOffMultiplier" value="2" />
</bean>

Now if a failure occurs, the configured delay isn't used on the first redelivery attempt. Instead redelivery takes place immediately after the failure occured. This sounds odd to me, since the property is even called initialRedeliveryDelay . I know that maybe this was caused due to the following issue:

https://issues.apache.org/activemq/browse/AMQ-1032

In my opinion immediately redelivery isn't very useful in most cases. If a failure occurs, it won't be fixed some ms later . So my suggestion is (in respect to AMQ-1032) to have two options:

initialRedeliveryDelay
redeliveryDelay

So both configurations would be possible. For AMQ-1032 just configure initialRedeliveryDelay to 0. Otherwise just take the redeliveryDelay as initialRedeliveryDelay to get what I want .

Does this make sense?

Comments

[Claus Ibsen]

I have just tried an unit test and it sleeps for 30 seconds.
[ main] RedeliveryPolicy DEBUG Sleeping for: 30000 millis until attempting redelivery

And then on the 2nd
[ main] RedeliveryPolicy DEBUG Sleeping for: 60000 millis until attempting redelivery

Your route doesn't do anything with the received message from activemq:NEW. How do you actually trigger the failure and the redelivery?

[Claus Ibsen]

I have tried with a jms example and it works in the unit tests in the source code.

Are you sure you have Camel 1.4-SNAPSHOT in ActiveMQ where you run your project?

[Claus Ibsen]

Hi Claus.

Meanwhile I've realized that the problem is a misunderstanding on my side what camel is supposed to do.
I thought I can configure the redelivery behavior of ActiveMQ within the camelContext as part of the acivemq.xml configuration on a per destination basis. What I haven't mentioned is, that my listener/consumer for the queue isn't configured within camel or activemq but in a separate spring baked application. What I now have realized is, that I have to consume a message via camel to get it's routing/mediation support (including redelivery
features) - now it's crystal clear to me (or not?). So I configured redeliveryPolicy on a ActiveMQConnectionFactory on my spring baked application and the settings are working fine now.

Sorry for mixing things up.

Thanks for you help.

Regards,
Veit

[Veit Guna]

Hi.

Sorry for reopening but the redeliveryDelay problem still exists. But it seems no camel problem then because I'm not using camel anymore, instead the redeliveryPolicy on the AQConnectionFactory.

So, should that be moved to the AQM project then? There I'm using AMQ 5.1.0 (Camel 1.3).

[Claus Ibsen]

Hi Veit

Yes please open a ticket on the ActiveMQ project if its solely an ActiveMQ problem.
http://issues.apache.org/activemq/browse/AMQ

[Claus Ibsen]

Veit. I managed to move the issue from Camel to ActiveMQ project.

[Claus Ibsen]

Veit. Can you add your new configuration etc. for the ActiveMQ that demonstrates the problem? I am sure the ActiveMQ comitters want to see your configuration file and what the problem is you have.

[Veit Guna]

Thanks Claus.

Sure, here's my applicationContext.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core-5.1.0.xsd">

<!-- ActiveMQ destinations to use -->
<bean id="destination" class="org.apache.activemq.command.ActiveMQQueue" autowire="constructor">
<constructor-arg value="NEW" />
</bean>

<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>discovery:multicast://test</value>
</property>
<property name="redeliveryPolicy">
<ref local="redeliveryPolicy" />
</property>
</bean>
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="initialRedeliveryDelay" value="10000" />
<property name="maximumRedeliveries" value="4" />
<property name="useExponentialBackOff" value="true" />
<property name="backOffMultiplier" value="2" />
</bean>

<!-- Spring JMS Template -->
<bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory">
<!-- lets wrap in a pool to avoid creating a connection per send -->
<bean class="org.springframework.jms.connection.SingleConnectionFactory">
<property name="targetConnectionFactory">
<ref local="jmsFactory" />
</property>
</bean>
</property>
</bean>

<bean id="consumerJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="jmsFactory"/>
</bean>

<!-- and this is the message listener container -->
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactory"/>
<property name="destination" ref="destination"/>
<property name="messageListener" ref="messageListener" />
<property name="concurrentConsumers" value="100" />
<property name="maxMessagesPerTask" value="1" />
<property name="sessionTransacted" value="true" />
</bean>
<!-- this is the Message Driven POJO (MDP) -->
<bean id="messageListener" class="com.foo.bar.jms.Consumer" />

</beans>

The messageListener will create a RuntimeException on the onMessage method to produce an error. The first redelivery attempt occurs immediately, without respect to the initialRedeliveryDelay.

[Dima]

Redelivery policy doesn't work with Spring's org.springframework.jms.listener.DefaultMessageListenerContainer at all

That happens, because DefaultMessageListenerContainer closes the consumer, session and connection, and do the reconnect if message listener throws exception, and of course previous delivery timepoint is empty in the new consumer.

It happens, because DefaultMessageListenerContainer doesn't know what a cause of JMSException. Maybe it was disconnection... and ConnectionFactory haven't reconnection logic...

I wrote my own simple MessageListenerContainer, that doesn't do reconnection if JMSException is thrown, because ActiveMQConnectionFactory has reconnection logic

[Veit Guna]

I'm sorry to correct you, but it IS working with the above configuration. All parameters I change within the redeliveryPolicy bean definition affecting the redelivery within the sping bean! The only thing that doesn't work correctly (or needs enhancement) is the initial redelivery.

[Dima]

I don't know how do you reach that, but if I use DefaultMessageListenerContainer all redeliveries happens immediately, not first only.

If any Execption in MessageListener happens it always closes the connnection

DefaultMessageListenerContainer.java

private class AsyncMessageListenerInvoker implements SchedulingAwareRunnable {

....

    catch (Throwable ex) {
    clearResources();

...


    }

...
}


private void clearResources() {
    if (sharedConnectionEnabled()) {
    synchronized (sharedConnectionMonitor) {
	JmsUtils.closeMessageConsumer(this.consumer);
	JmsUtils.closeSession(this.session);
        }
    }
    else {
	JmsUtils.closeMessageConsumer(this.consumer);
	JmsUtils.closeSession(this.session);
    }
	this.consumer = null;
	this.session = null;
}

[Veit Guna]

I don't know. Perhaps configuration, ActiveMQ 5.1.0 or Spring 2.5.5. Perhaps someone fixed/enhanced something .

[Veit Guna]

@Dima:

Now, I can confirm your findings! After I set the jmsTransactionManager in my config (not just sessionTransacted=true like before) I encounter immediate redelivery without delay/backoffs, too:

<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactory"/>
<property name="destination" ref="destination"/>
<property name="messageListener" ref="messageListener" />
<property name="concurrentConsumers" value="2" />
<property name="maxMessagesPerTask" value="1" />
<property name="sessionTransacted" value="true" />
<property name="transactionManager" ref="jmsTransActionManager" />
</bean>

After I set this, the MesageListener also checks connections actively - which is very noisy. So in the end: is this all expected behavior? I'm a bit confused...