Details
Major Description
Problem Description:
MRG LVQ becomes unstable when msg TTLs are used and a message expires. Sometimes when messages expire, it seems to upset the LVQ mechanism. Eg. Once a message with key 'X' has expired, any subsequent messages sent to the LVQ with key 'X' are expired immediately, and never made available to client applications. Once an LVQ begins exhibiting this behaviour, This behaviour continues until the broker is restarted. At this point, another side effect is that the msgDepth and byteDepth properties on an LVQ do not always agree, i.e, when msgDepth is zero, byteDepth is not zero.
How Reproducible:
Always:
Steps to Reproduce:
- Create a queue with queue ordering as lvq or lvq-no-browse
- Build and run attached producer for 1 minute
- Stop the producer for a minute to allow messages to expire
- Use qpid-tool to monitor the queue depth and wait for the message to be dequeued (usually takes about 10 minutes)
- Once message has expired, restart the producer
- Watch queue properties with qpid-tool
Actual Results:
The queue depth is always 0, but the byte depth is not. Occasionally, one will get the following error message from the producer:
Unexpected exception: Attempted size underflow on dequeue
Expected Results:
The queue depth should not be 0 (at least till the time expired messages are purged). Also, when queue depth is 0, byte depth should be 0.
Additional Information:
The root cause seems to be that the lvq object in the Queue that holds mapping from key to messages (Queue::lvq) is not cleared when messages are expired (Queue::purgeExpired), which leads to incorrect accounting when the next message arrives in the queue with the same LVQ key as the message that expired.