Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.0.0
-
None
Description
Support for streaming messages (commit c9262728) introduced locking between message receive and message send.
Before streaming:
- message_receive creates the message buffer list with no locks
- message_send sends messages with no locks
- When all copies of the message are sent then the last sender deletes the message content including all buffers
With streaming:
- message_receive takes the content lock (per buffer) as each buffer is added
- message_send takes the content lock (per buffer) as each buffer is consumed. This happens once for each message copy.
- message_send possibly frees the buffer if all message copies have sent that buffer.
- When all copies of the message are sent then the last sender deletes the message content. All buffers are already freed.
There may be a problem with all those locks:
- Lock ownership If the message is being streamed to N destinations then the buffer lock will have contention from N I/O threads and the mutex ownership will surely bounce from thread to thread.
- Lock per buffer add/remove If a 1 Mbyte message is streamed to N destinations then the buffer lock will be taken 2000 times by message_receive and (N * 2000) times by message_send.
With some careful design lock usage could be greatly reduced or even eliminated.