Mesos
  1. Mesos
  2. MESOS-418

Add security and authentication support to Mesos (including integration with LDAP).

    Details

    • Type: Story Story
    • Status: Closed
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None

      Description

      The basic idea behind the proposal, is to add authorization/authentication support to Mesos. For example, Mesos should only allow authenticated frameworks to register and submit jobs. The plan is to leverage Kerberos/LDAP to add this support. We are also open to suggestions on how we can add support for security and auth in Mesos.

      Knowledge Prerequisite: C++

        Issue Links

          Activity

          Vinod Kone created issue -
          Vinod Kone made changes -
          Field Original Value New Value
          Labels gsoc gsoc2013 mentor c++ cloud gsoc gsoc2013 mentor
          Hide
          Zamar Edwin added a comment - - edited

          Hello,

          I am interested in this issue. Which skills are you looking for in a successful GSoC candidate? Thank you.


          Zamar

          Show
          Zamar Edwin added a comment - - edited Hello, I am interested in this issue. Which skills are you looking for in a successful GSoC candidate? Thank you. – Zamar
          Hide
          Vinod Kone added a comment -

          Glad you are interested! We are looking for candidates well versed in C++ and preferably security/auth domain.

          Show
          Vinod Kone added a comment - Glad you are interested! We are looking for candidates well versed in C++ and preferably security/auth domain.
          Hide
          Zamar Edwin added a comment - - edited

          Thank you very much for your response. I have moderate knowledge of C++
          that is growing everyday, but am not familiar with security/auth domain.
          How difficult would it be to acquire these skills on the job? I have very
          few commitments this summer and plan to contribute a significant amount of
          my time (40+ hours) per week to an open source project.


          Zamar

          Show
          Zamar Edwin added a comment - - edited Thank you very much for your response. I have moderate knowledge of C++ that is growing everyday, but am not familiar with security/auth domain. How difficult would it be to acquire these skills on the job? I have very few commitments this summer and plan to contribute a significant amount of my time (40+ hours) per week to an open source project. – Zamar
          Hide
          Vinod Kone added a comment -

          Sounds good. Definitely send in your proposal for the project. We select candidate(s) based on the strength of their proposal and their portfolio (if any).

          Show
          Vinod Kone added a comment - Sounds good. Definitely send in your proposal for the project. We select candidate(s) based on the strength of their proposal and their portfolio (if any).
          Hide
          Zamar Edwin added a comment -

          Will do. I will keep in contact and ask more questions as I do more research of the project. Thanks.

          Show
          Zamar Edwin added a comment - Will do. I will keep in contact and ask more questions as I do more research of the project. Thanks.
          Hide
          brian wickman added a comment -

          Here is a good introduction of the alphabet soup that is the security infrastructure we're investigating: http://www.kerberos.org/software/appskerberos.pdf

          Show
          brian wickman added a comment - Here is a good introduction of the alphabet soup that is the security infrastructure we're investigating: http://www.kerberos.org/software/appskerberos.pdf
          Hide
          Zamar Edwin added a comment -

          Thank you for the link. I have been reading through it since yesterday and following the links to specifications and other papers/videos. I have also been looking into LDAP resources. Are there any LDAP sources, relevent to this project, that you would recommend? Thanks.

          Show
          Zamar Edwin added a comment - Thank you for the link. I have been reading through it since yesterday and following the links to specifications and other papers/videos. I have also been looking into LDAP resources. Are there any LDAP sources, relevent to this project, that you would recommend? Thanks.
          Hide
          brian wickman added a comment -

          It's probably not necessary to spend too much time looking into LDAP, as that's more of a concern for the system administrators of Kerberos (specifically how the KDB backend is configured within their organization) and not the concern of Mesos. There is nothing that needs to tie the Mesos infrastructure to LDAP – the JIRA guidance is probably slightly off.

          Show
          brian wickman added a comment - It's probably not necessary to spend too much time looking into LDAP, as that's more of a concern for the system administrators of Kerberos (specifically how the KDB backend is configured within their organization) and not the concern of Mesos. There is nothing that needs to tie the Mesos infrastructure to LDAP – the JIRA guidance is probably slightly off.
          Hide
          Zamar Edwin added a comment -

          Thanks again. I am working on my proposal and am almost finished with the paper.

          Show
          Zamar Edwin added a comment - Thanks again. I am working on my proposal and am almost finished with the paper.
          Zamar Edwin made changes -
          Comment [ Thanks again. I am working on my proposal and am almost finished with the
          paper.


          ]
          Hide
          Zamar Edwin added a comment - - edited

          I have been working off of the proposal examples described here: http://www.booki.cc/gsocstudentguide/proposal-examples/. Is there anything, not in the examples, that you all would like to see in a successful proposal? Thank you.

          Show
          Zamar Edwin added a comment - - edited I have been working off of the proposal examples described here: http://www.booki.cc/gsocstudentguide/proposal-examples/ . Is there anything, not in the examples, that you all would like to see in a successful proposal? Thank you.
          Hide
          Vinod Kone added a comment -

          Hey Zamar. Those examples look pretty good to me. Some of the other things we would like to see any previous open source projects you have worked on (not necessarily related to security/auth).

          Show
          Vinod Kone added a comment - Hey Zamar. Those examples look pretty good to me. Some of the other things we would like to see any previous open source projects you have worked on (not necessarily related to security/auth).
          Hide
          Zamar Edwin added a comment -

          Thanks for your reply. I have not worked on open source projects before, however. So this would be my first experience.

          Show
          Zamar Edwin added a comment - Thanks for your reply. I have not worked on open source projects before, however. So this would be my first experience.
          Vinod Kone made changes -
          Assignee Vinod Kone [ vinodkone ]
          Hide
          Vinod Kone added a comment -

          Assigning this to Ilim Ugur as he is taking this on as part of GSOC13.

          Show
          Vinod Kone added a comment - Assigning this to Ilim Ugur as he is taking this on as part of GSOC13.
          Vinod Kone made changes -
          Assignee Vinod Kone [ vinodkone ] Ilim Ugur [ ilim ]
          Hide
          Ilim Ugur added a comment -

          Hey! Just wanted to introduce myself, as I will be working on this issue as my GSoC '13 project. A project wiki/blog is soon to be set up, and I will share the link to it as soon as it is available.

          Show
          Ilim Ugur added a comment - Hey! Just wanted to introduce myself, as I will be working on this issue as my GSoC '13 project. A project wiki/blog is soon to be set up, and I will share the link to it as soon as it is available.
          Hide
          Benjamin Hindman added a comment -

          Awesome! Welcome and looking forward to authentication support!

          Show
          Benjamin Hindman added a comment - Awesome! Welcome and looking forward to authentication support!
          Hide
          Vinod Kone added a comment -

          Added a wiki page at https://cwiki.apache.org/confluence/display/MESOS/Authentication%2C+Authorization+and+Security+support

          Ilim Ugur would be updating the page with design discussions etc, as he makes progress.

          Show
          Vinod Kone added a comment - Added a wiki page at https://cwiki.apache.org/confluence/display/MESOS/Authentication%2C+Authorization+and+Security+support Ilim Ugur would be updating the page with design discussions etc, as he makes progress.
          Gavin made changes -
          Issue Type New Feature [ 2 ] Story [ 16 ]
          Show
          Chris Aniszczyk added a comment - Ilim Ugur , how are things coming along? https://cwiki.apache.org/confluence/display/MESOS/Authentication+Support
          Benjamin Mahler made changes -
          Link This issue blocks MESOS-338 [ MESOS-338 ]
          Hide
          Ilim Ugur added a comment -

          The project, although is still ongoing, could not be finalized until the end of the coding period of Google Summer of Code. Still, the project is to continue, according to the roadmap on the wiki page at https://cwiki.apache.org/confluence/display/MESOS/Authentication+Support

          Show
          Ilim Ugur added a comment - The project, although is still ongoing, could not be finalized until the end of the coding period of Google Summer of Code. Still, the project is to continue, according to the roadmap on the wiki page at https://cwiki.apache.org/confluence/display/MESOS/Authentication+Support
          Hide
          David WEI added a comment -

          Hi, my name is David. I'd like to do sth for this open source project.
          I am an experienced software engineer working on network security in Bay Area.
          Here are my draft about general use cases, the main idea is to add into Mesos a
          cyrus-sasl client which will send the authentication info to security server to get verification.
          Your comments or updates are welcome.

          General use cases/test cases

          1, Mesos users(framework application) or slaves register(i.e. name and password) to security server. This may be integrated into central secruity management which is outside of Mesos. In the unit tests, we may use open source SASL server, such as cyrus-sasl2 in Ubuntu.

          2, When Mesos Master gets framework application or slave register(resource allocation)request, based on security setting of Mesos, there are following cases

          1) Anonymous allowed and no authentication info in the request. This is compatible with current implementation.

          2) Authentication support. Extract authentication info from the request, and send to configured security
          server by Cyrus SASL interface. If get successful authentication, then continue to do framework or slave register ,else reject the register request.

          Note: Considering the performance impact introduced by the delay of this authentication request and response communication,
          one option is to present a local authenticated user table in Master node. It works as a cache. For each authentication, the local table will be looked up firstly, if not found, then communicate with security server. After get successful authentication, the user authid and a timestamp is inserted into the local table. Then within a configured period(i.e. 24 hours), the following register request from this user will get permit from local table.
          For the Master failure recovery, the local table will be re-buit on the received re-register requests.

          Show
          David WEI added a comment - Hi, my name is David. I'd like to do sth for this open source project. I am an experienced software engineer working on network security in Bay Area. Here are my draft about general use cases, the main idea is to add into Mesos a cyrus-sasl client which will send the authentication info to security server to get verification. Your comments or updates are welcome. General use cases/test cases 1, Mesos users(framework application) or slaves register(i.e. name and password) to security server. This may be integrated into central secruity management which is outside of Mesos. In the unit tests, we may use open source SASL server, such as cyrus-sasl2 in Ubuntu. 2, When Mesos Master gets framework application or slave register(resource allocation)request, based on security setting of Mesos, there are following cases 1) Anonymous allowed and no authentication info in the request. This is compatible with current implementation. 2) Authentication support. Extract authentication info from the request, and send to configured security server by Cyrus SASL interface. If get successful authentication, then continue to do framework or slave register ,else reject the register request. Note: Considering the performance impact introduced by the delay of this authentication request and response communication, one option is to present a local authenticated user table in Master node. It works as a cache. For each authentication, the local table will be looked up firstly, if not found, then communicate with security server. After get successful authentication, the user authid and a timestamp is inserted into the local table. Then within a configured period(i.e. 24 hours), the following register request from this user will get permit from local table. For the Master failure recovery, the local table will be re-buit on the received re-register requests.
          Hide
          Benjamin Hindman added a comment -

          Hi David, have you had a chance to look at how we've done authentication using Cyrus SASL thus far? The next steps are probably (1) integrating in Kerberos and (2) adding authentication support to the slaves (similar to how we did it for the schedulers) and (3) creating some form of authorization (pluggable by LDAP most likely).

          Show
          Benjamin Hindman added a comment - Hi David, have you had a chance to look at how we've done authentication using Cyrus SASL thus far? The next steps are probably (1) integrating in Kerberos and (2) adding authentication support to the slaves (similar to how we did it for the schedulers) and (3) creating some form of authorization (pluggable by LDAP most likely).
          Vinod Kone made changes -
          Link This issue is related to MESOS-803 [ MESOS-803 ]
          Vinod Kone made changes -
          Link This issue is related to MESOS-804 [ MESOS-804 ]
          Hide
          Adam B added a comment -

          I am happy to see framework authentication via SASL (MESOS-704) in 0.15.0. Has anybody made progress on the other security tasks? We would like to begin work soon on (1) Kerberos integration, (2) slave authentication (MESOS-804), (3) pluggable (LDAP) authorization, and (4) SSL data encryption. If nobody objects, I will create child JIRAs for 1, 3, and 4 and we can begin work on them.

          Show
          Adam B added a comment - I am happy to see framework authentication via SASL ( MESOS-704 ) in 0.15.0. Has anybody made progress on the other security tasks? We would like to begin work soon on (1) Kerberos integration, (2) slave authentication ( MESOS-804 ), (3) pluggable (LDAP) authorization, and (4) SSL data encryption. If nobody objects, I will create child JIRAs for 1, 3, and 4 and we can begin work on them.
          Hide
          Vinod Kone added a comment -

          Not that I'm aware of. Feel free to create the JIRAs. It would be great if we can have any major design discussions on the respective JIRAs.

          Show
          Vinod Kone added a comment - Not that I'm aware of. Feel free to create the JIRAs. It would be great if we can have any major design discussions on the respective JIRAs.
          Hide
          Dave Lester added a comment -

          Adam B Sounds great! As Vinod said, it would be great to create JIRA for each of these tasks.

          David WEI Ping, in case you're still interested in contributing to this.

          Show
          Dave Lester added a comment - Adam B Sounds great! As Vinod said, it would be great to create JIRA for each of these tasks. David WEI Ping, in case you're still interested in contributing to this.
          Hide
          David WEI added a comment - - edited

          Sure, I am. It is a good idea to have major design discussions for the JIRAs.

          Vinod, could you share some info of the major design(or high level design) of MESOS-704?

          It will be a foundation for further tasks related to security and authentication.

          Show
          David WEI added a comment - - edited Sure, I am. It is a good idea to have major design discussions for the JIRAs. Vinod, could you share some info of the major design(or high level design) of MESOS-704 ? It will be a foundation for further tasks related to security and authentication.
          Adam B made changes -
          Link This issue relates to MESOS-907 [ MESOS-907 ]
          Adam B made changes -
          Link This issue is related to MESOS-907 [ MESOS-907 ]
          Adam B made changes -
          Link This issue is related to MESOS-911 [ MESOS-911 ]
          Adam B made changes -
          Link This issue is related to MESOS-910 [ MESOS-910 ]
          Adam B made changes -
          Link This issue relates to MESOS-907 [ MESOS-907 ]
          Hide
          Adam B added a comment -

          Added links to new JIRAs:
          MESOS-907 Add Kerberos Authentication support
          MESOS-910 Add encryption support for master/slave/framework channels
          MESOS-911 Add pluggable authorization interface
          Please comment on these JIRAs if you have any relevant ideas/suggestions.

          Show
          Adam B added a comment - Added links to new JIRAs: MESOS-907 Add Kerberos Authentication support MESOS-910 Add encryption support for master/slave/framework channels MESOS-911 Add pluggable authorization interface Please comment on these JIRAs if you have any relevant ideas/suggestions.
          Vinod Kone made changes -
          Link This issue is related to MESOS-911 [ MESOS-911 ]
          Hide
          Benjamin Hindman added a comment -

          This was far too broad of an issue, perhaps even too broad for an epic! There have been a lot of authentication and authorization work since, so I'm going to remove this in favor of those.

          Show
          Benjamin Hindman added a comment - This was far too broad of an issue, perhaps even too broad for an epic! There have been a lot of authentication and authorization work since, so I'm going to remove this in favor of those.
          Benjamin Hindman made changes -
          Status Open [ 1 ] Closed [ 6 ]
          Assignee Ilim Ugur [ ilim ]
          Resolution Invalid [ 6 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Vinod Kone
            • Votes:
              7 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development