[DISPATCH-369] investigate excursions in memory usage

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Fix Version/s: Backlog
• Component/s: Router Node
• Labels: memory-bug
• Assignee: Michael Goulish
• Reporter: Michael Goulish
• Created: 08/Jun/16 05:16
• Updated: 11/Oct/19 14:36

Description

I don't know if this is a bug or not. I'm Jirifying it as a way of remembering an interesting behavior that my testing has shown, so that I can continue developing the testing and come back to this later.

...

While measuring router memory usage under varying message rate and number of senders – when I run the same test multiple times, I am occasionally (about 1 in 4 times or so) seeing a test in which memory usage is much higher than the others.

For example:
In this test:

{ straight-through topology ( 1 sender --> 1 address --> 1 receiver ) 200 senders 200 messages per second 100 bytes per message }

I record router memory usage at the point when all receivers are just hitting 10,000 messages. (This is because it grows – see previous JIRA.)

In three iterations I get the following memory usage:

66 MB
63 MB
181 MB

Something similar, but less drastic, happened occasionally at lower levels in the test.

In this case, this is a tripling of memory usage for the same scenario. I doubt that this is the result of slightly different timing in a block allocation of data structures. What just happened?

Start by investigating with "qdstat -m" and see if that shows some or all of the difference.