[MESOS-3332] Support HTTP Pipelining in libprocess (http::post)

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.26.0
• Component/s: libprocess
• Labels: twitter
• Epic Link: Scheduler API v1 improvements
• Sprint: Twitter Mesos Q3 Sprint 4, Twitter Mesos Q3 Sprint 5, Twitter Mesos Q3 Sprint 6
• Story Points: 8
• Assignee: Benjamin Mahler
• Reporter: Anand Mazumdar
• Created: 28/Aug/15 20:20
• Updated: 06/Oct/15 00:13
• Resolved: 06/Oct/15 00:13

Description

Currently , http::post in libprocess, does not support HTTP pipelining. Each call as of know sends in the Connection: close header, thereby, signaling to the server to close the TCP socket after the response.

We either need to create a new interface for supporting HTTP pipelining , or modify the existing http::post to do so.

This is needed for the Scheduler/Executor library implementations to make sure "Calls" are sent in order to the master. Currently, in order to do so, we send in the next request only after we have received a response for an earlier call that results in degraded performance.