[MESOS-8668] Transition libprocess on Windows to use the Thread Pool API

• Type: Epic
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: libprocess
• Labels: libprocess windows

Environment

Windows 10

• Epic Name: libprocess-iocp
• Target Version/s: 1.7.0
• Assignee: Unassigned
• Reporter: Andrew Schwartzmeyer
• Shepherd: Andrew Schwartzmeyer
• Due: 24/Apr/18
• Created: 13/Mar/18 22:57
• Updated: 28/Jun/18 18:44
• Resolved: 28/Jun/18 18:44

Description

This epic covers the work necessary to use the Windows Thread Pool API instead of libev or libevent on Windows.

The design of libprocess closely mimics the built-in Thread Pool API, using I/O Completion Ports for asynchronous I/O. It makes sense to use this API because the semantics of libprocess map 1-1 to the Windows async asynchronous support.

Transitioning to IOCP is necessary to implement non-blocking pipes, as the existing eventing libraries (e.g. libevent) only implement asynchronous sockets on Windows; but there are many instances of Mesos requiring asynchronous pipes too. As such, the existing use of libevent is fundamentally broken.

Specifically, this was first discovered in test HungDockerTest.ROOT_DOCKER_InspectHungDuringPull, but is also evidenced through IOTest.Read and others which use non-blocking pipes.

The alternative eventing libraries libev and libuv were considered, but we are choosing not to use them as they do not implement non-blocking pipes sufficiently for Windows. libev is not easy to build on Windows, and does not support IOCP. While libevent uses IOCP for reading and writing sockets, it uses select() to detect and dispatch events (which results in poor performance), and does not use support IOCP for pipes. libuv claims to support IOCP on Windows, but they do not support named pipes and only supportes synchronous pipes. Moreover, it is simpler to use the Thread Pool API in libprocess directly, instead of using a library to abstract over that API, as libprocess already provides the abstraction we're seeking for Mesos.

Additionally, by using the built-in Windows IOCP support, we can eventually support SSL using the Windows crypto libraries, instead of requiring OpenSSL on Windows (the current case for libprocess).

Issue Links

• incorporates: MESOS-8682 Remove remaining CRT functions stout on Windows
• is related to: MESOS-5815 Port libprocess io_tests.cpp
• is related to: MESOS-3529 Transition Windows builds from beta version of libevent to stable.
• supercedes: MESOS-3526 Transition Windows from libevent to libev