[PROTON-1687] Improve efficiency of the transport output buffer

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: proton-c-0.18.0
• Fix Version/s: proton-c-0.19.0
• Component/s: proton-c
• Labels: perf
• Assignee: Andrew Stitcher
• Reporter: Ted Ross
• Created: 15/Nov/17 14:10
• Updated: 24/Dec/17 16:38
• Resolved: 11/Dec/17 19:06

Description

As the size of the transport output buffer grows due to production of deliveries sent through pn_link_send, the amount of time that the CPU spends in memmove becomes a problem.

It appears (dispatcher.c - pn_dispatcher_output) that whenever a block of data is removed from the buffer, the remainder of the data in the buffer is shifted down using memmove. If the buffer contains a large amount of data (100s of megabytes), the sending process may spend several minutes doing almost nothing but running memmove.

Some data:
(credit window of 250, unlimited session window, 1 MB payload)

• 10 messages, .251 sec, 40 deliveries/sec
• 20 messages, 1.267s, 16 dlv/s
• 40 messages, 6s, 7 dlv/s
• 80 messages, 28s, 3 dlv/s
• 160 messages, 138s, ~1 dlv/s

Issue Links

• relates to: PROTON-1521 [c proactor] sender stalls responding to Flow opening incomingWindow
• relates to: PROTON-659 if protons internal buffer gets large, performance can suffer