Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-8668

Transition libprocess on Windows to use the Thread Pool API

    XMLWordPrintableJSON

    Details

    • Type: Epic
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: libprocess
    • Environment:

      Windows 10

    • Epic Name:
      libprocess-iocp
    • Target Version/s:

      Description

      This epic covers the work necessary to use the Windows Thread Pool API instead of libev or libevent on Windows.

      The design of libprocess closely mimics the built-in Thread Pool API, using I/O Completion Ports for asynchronous I/O. It makes sense to use this API because the semantics of libprocess map 1-1 to the Windows async asynchronous support.

      Transitioning to IOCP is necessary to implement non-blocking pipes, as the existing eventing libraries (e.g. libevent) only implement asynchronous sockets on Windows; but there are many instances of Mesos requiring asynchronous pipes too. As such, the existing use of libevent is fundamentally broken.

      Specifically, this was first discovered in test HungDockerTest.ROOT_DOCKER_InspectHungDuringPull, but is also evidenced through IOTest.Read and others which use non-blocking pipes.

      The alternative eventing libraries libev and libuv were considered, but we are choosing not to use them as they do not implement non-blocking pipes sufficiently for Windows. libev is not easy to build on Windows, and does not support IOCP. While libevent uses IOCP for reading and writing sockets, it uses select() to detect and dispatch events (which results in poor performance), and does not use support IOCP for pipes. libuv claims to support IOCP on Windows, but they do not support named pipes and only supportes synchronous pipes. Moreover, it is simpler to use the Thread Pool API in libprocess directly, instead of using a library to abstract over that API, as libprocess already provides the abstraction we're seeking for Mesos.

      Additionally, by using the built-in Windows IOCP support, we can eventually support SSL using the Windows crypto libraries, instead of requiring OpenSSL on Windows (the current case for libprocess).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                andschwa Andrew Schwartzmeyer
                Shepherd:
                Andrew Schwartzmeyer
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1,008h
                  1,008h
                  Remaining:
                  Remaining Estimate - 1,008h
                  1,008h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified