[QPID-3316] flow-to-disk limit policy does not flow the whole message to disk

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.8, 0.9, 0.10
• Component/s: C++ Broker
• Labels: flow-to-disk limit-policy qpid
• Assignee: Unassigned
• Reporter: Sergey Zhemzhitsky
• Created: 22/Jun/11 13:23
• Updated: 12/Mar/13 18:08
• Resolved: 12/Mar/13 18:08

Description

Here is part of the mailing list

Hi there,

I'm using qpid 0.8 and I'm trying to configure queue to use the limit-policy parameter without luck.
Here is the command to create a queue

qpid-config add queue myQueue --durable --file-size=256
--file-count=96 --max-queue-count=100 --limit-policy=flow-to-disk

Here is output of the top command after qpid starts

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28146 qpidd 19 0 123m 5596 4104 S 0.0 0.3 0:00.00 qpidd

As you can see it uses ~5.6mb of memory.

After I start to send non-persistent messages (size of each message is 32 bytes) without consumption the output of the top command is the following:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28146 qpidd 16 0 2541m 1.8g 3520 S 49.0 94.1 0:41.75 qpidd

Now qpid consumes 1.8Gb and it continues to consume more.

I'm wondering why so? How to make the queue to use the limit-policy parameter correctly?

Here is the answer from Gordon Sim:

That policy allows the body of the message to be released from memory, but the headers etc will still be held in memory. For small messages (such as the 32 byte message you are sending), this policy will not really do anything useful.

Your use case really requires a proper paging solution which qpidd does not yet support.