Details

    • Epic Name:
      SSL
    • Target Version/s:

      Description

      Currently all the messages that flow through the Mesos cluster are unencrypted
      making it possible for intruders to intercept and potentially control your task.
      We plan to add encryption support by adding SSL/TLS support to libprocess, the
      low-level communication library that Mesos uses for all network communication
      between Mesos components.
      As a first step, we should replace the hand-coded http code in libprocess with a
      standard library, ensuring that any mesos custom code like routing remains.
      Then, transition to https should be easier.

      Road map to SSL

      1. Isolate libev dependencies to a manageable set of implementing files.
        1. MESOS-1912 Decouple libev from clock implementation
        2. MESOS-1914 Decouple libev from connection handling (use io::poll() instead of individual watchers)
        3. MESOS-1952 Abstract network logic into socket class: connect()
        4. MESOS-1954 Abstract network logic into socket class: read()/write()
        5. MESOS-1953 Abstract network logic into socket class: connection events (connected(), closed(), writable(), readable())
        6. MESOS-2119 Add Socket tests
        7. (MESOS-XXXX Libev backed Socket)
      2. Provide alternative implementation with libevent.
        1. MESOS-2106 Enable libevent backed libprocess with configure flag.
        2. MESOS-2107 Create libevent-backed clock implementation
        3. MESOS-2133 Create libevent-backed poll implementation
        4. MESOS-1911 Create libevent-backed socket implementation
      3. Enable SSL
        1. MESOS-2108 Add configure flag or environment variable to enable SSL/libevent Socket
        2. MESOS-2109 Introduce socket factory
        3. MESOS-1913 Create libevent/SSL-backed Socket implementation
        4. MESOS-2085 Add support encrypted and non-encrypted communication in parallel for cluster upgrade

        Issue Links

          Issues in Epic

            Activity

            Hide
            benjaminhindman Benjamin Hindman added a comment -

            Which "libhttp" did you have in mind?

            Show
            benjaminhindman Benjamin Hindman added a comment - Which "libhttp" did you have in mind?
            Hide
            adam-mesos Adam B added a comment -

            I haven't investigated the various libraries much yet. A quick stackoverflow search suggests the following:
            Mongoose (GPLv2): http://code.google.com/p/mongoose/
            libmicrohttpd (LGPLv2.1): http://www.gnu.org/software/libmicrohttpd/
            lighttpd (BSD): http://www.lighttpd.net/
            Pion (Boost): https://github.com/cloudmeter/pion (Splunk)
            boost-asio http server examples (): http://www.boost.org/doc/libs/1_43_0/doc/html/boost_asio/examples.html
            I believe all of them can support SSL/TLS, often through an external library like gnutls.
            Suggestions/comments are welcome, especially if you can explain what other features/functionality will be needed.

            Show
            adam-mesos Adam B added a comment - I haven't investigated the various libraries much yet. A quick stackoverflow search suggests the following: Mongoose (GPLv2): http://code.google.com/p/mongoose/ libmicrohttpd (LGPLv2.1): http://www.gnu.org/software/libmicrohttpd/ lighttpd (BSD): http://www.lighttpd.net/ Pion (Boost): https://github.com/cloudmeter/pion (Splunk) boost-asio http server examples (): http://www.boost.org/doc/libs/1_43_0/doc/html/boost_asio/examples.html I believe all of them can support SSL/TLS, often through an external library like gnutls. Suggestions/comments are welcome, especially if you can explain what other features/functionality will be needed.
            Hide
            nnielsen Niklas Quarfot Nielsen added a comment -

            How about keeping the current HTTP handling, creating an SSL/TLS (and plain) connection abstraction in stout (built on openssl) and create a new HttpsConnection SocketProcess in libprocess?

            This change will probably result in many subsequent patches, but would be one way to go. All instances of socket calls would need to be updated.

            Show
            nnielsen Niklas Quarfot Nielsen added a comment - How about keeping the current HTTP handling, creating an SSL/TLS (and plain) connection abstraction in stout (built on openssl) and create a new HttpsConnection SocketProcess in libprocess? This change will probably result in many subsequent patches, but would be one way to go. All instances of socket calls would need to be updated.
            Hide
            nnielsen Niklas Quarfot Nielsen added a comment -

            I think it makes sense to think in terms of different low or middle layer transports which we could capture connection life-cycles and network send/receive primitives in a much explicit manner than currently in libprocess. If that sounds reasonable to use, I will go ahead and create a ticket for such an abstraction and move this discussion to a subtask of the new one.

            Thoughts?

            Show
            nnielsen Niklas Quarfot Nielsen added a comment - I think it makes sense to think in terms of different low or middle layer transports which we could capture connection life-cycles and network send/receive primitives in a much explicit manner than currently in libprocess. If that sounds reasonable to use, I will go ahead and create a ticket for such an abstraction and move this discussion to a subtask of the new one. Thoughts?
            Hide
            nnielsen Niklas Quarfot Nielsen added a comment -

            New ticket on connection abstraction: https://issues.apache.org/jira/browse/MESOS-1330

            Show
            nnielsen Niklas Quarfot Nielsen added a comment - New ticket on connection abstraction: https://issues.apache.org/jira/browse/MESOS-1330
            Hide
            nnielsen Niklas Quarfot Nielsen added a comment -

            Followed the convention of only starting progress (and ownership) when a shepherd is found. Still, I have spent a ton of time on this and would still like to chime in/own this.

            Show
            nnielsen Niklas Quarfot Nielsen added a comment - Followed the convention of only starting progress (and ownership) when a shepherd is found. Still, I have spent a ton of time on this and would still like to chime in/own this.
            Hide
            benjaminhindman Benjamin Hindman added a comment -

            Hopefully soon we'll have the ACCEPTED status and then we can reassign to you + a shepherd (we know have the Shepherd field too, click 'Edit' above and scroll down and look for 'Shepherd'!).

            Show
            benjaminhindman Benjamin Hindman added a comment - Hopefully soon we'll have the ACCEPTED status and then we can reassign to you + a shepherd (we know have the Shepherd field too, click 'Edit' above and scroll down and look for 'Shepherd'!).
            Hide
            t1ckt0ck Scott Clasen added a comment -

            Just want to ping here for status, would love to see this get moving. Thanks!

            Show
            t1ckt0ck Scott Clasen added a comment - Just want to ping here for status, would love to see this get moving. Thanks!
            Hide
            nnielsen Niklas Quarfot Nielsen added a comment -

            Hi Scott Clasen - this ties into the connection/transport abstraction work Joris Van Remoortere is current doing. When libevent is supported as an event manager, enabling SSL is straight forward.

            Show
            nnielsen Niklas Quarfot Nielsen added a comment - Hi Scott Clasen - this ties into the connection/transport abstraction work Joris Van Remoortere is current doing. When libevent is supported as an event manager, enabling SSL is straight forward.
            Hide
            adam-mesos Adam B added a comment -

            This Epic/feature is our #1 blocker for Mesos 0.23.0. Upgraded its priority to Blocker.

            Show
            adam-mesos Adam B added a comment - This Epic/feature is our #1 blocker for Mesos 0.23.0. Upgraded its priority to Blocker.
            Hide
            adam-mesos Adam B added a comment -

            Required tasks have been completed. Resolving this Epic.

            Show
            adam-mesos Adam B added a comment - Required tasks have been completed. Resolving this Epic.

              People

              • Assignee:
                jvanremoortere Joris Van Remoortere
                Reporter:
                adam-mesos Adam B
                Shepherd:
                Benjamin Hindman
              • Votes:
                4 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development