[PROTON-888] allocate_alias linear search becomes slow at scale

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: proton-c-future
• Component/s: proton-c
• Labels: perf
• Assignee: Clifford Jansen
• Reporter: Michael Goulish
• Created: 18/May/15 18:27
• Updated: 05/Dec/17 14:52

Description

Testing that I have done recently goes to large scale on number of sessions per connection. I noticed that the test was slowing down rapidly over time, in terms of how many sessions were being established per unit time.

The function allocate_alias in file transport.c uses a linear search through an array to find the next available channel number for a session (or the next available handle number for a link). In a usage scenario like mine in which many sessions will be established, this becomes very slow as the array fills up.

At the beginning of my test, this function is too fast to measure. By the end, it is using more than 82 milliseconds per call. Overall, this function alone is contributing more than 20 seconds to my 3-minute test.

This is not an unrealistic scenario – we already have one potential customer who is interested in going to this kind of scale. (Which is why I was doing this test.)

Maybe we can find an implementation that does not slow down the common scale, and yet behaves better at the high end.