HTTP 1.1 Pipelining describes a mechanism by which multiple HTTP request can be performed over a single socket. The requirement here is that responses should be send in the same order as requests are being made.
Libprocess has some mechanisms built in to deal with pipelining when multiple HTTP requests are made, it is still, however, possible to create a situation in which responses are scrambled respected to the requests arrival.
Consider the situation in which there are two libprocess processes, processA and processB, each running in a different thread, thread2 and thread3 respectively. The ProcessManager runs in thread1.
processA is of type ProcessA which looks roughly as follows:
processB is from type ProcessB which is just like ProcessA but routes "bar" instead of "foo".
The situation in which the bug arises is the following:
- Two requests, one for "http://server_uri/(1)/foo" and one for "http://server_uri/(2)//bar" are made over the same socket.
- The first request arrives to ProcessManager::handle which is still running in thread1. This one creates an HttpEvent and delivers to the handler, in this case processA.
- ProcessManager::deliver enqueues the HTTP event in to the processA queue. This happens in thread1.
- The second request arrives to ProcessManager::handle which is still running in thread1. Another HttpEvent is created and delivered to the handler, in this case processB.
- ProcessManager::deliver enqueues the HTTP event in to the processB queue. This happens in thread1.
- Thread2 is blocked, so processA cannot handle the first request, it is stuck in the queue.
- Thread3 is idle, so it picks up the request to processB immediately.
- ProcessBase::visit(HttpEvent) is called in thread3, this one in turn dispatches the response's future to the HttpProxy associated with the socket where the request came.
At the last point, the bug is evident, the request to processB will be send before the request to processA even if the handler takes a long time and the processA::bar() actually finishes before. The responses are not send in the order the requests are done.
The following is a test which successfully reproduces the issue: