Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1670

puppet: Support Kerberos authentication on Hadoop component web GUIs

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      Support configuration of Kerberos authentication on Hadoop component web GUIs. Also introduce support for trocla for randomly generating secrets that are stored on the master, don't change after creation and can be the same across hosts.

        Activity

        Hide
        evans_ye Evans Ye added a comment -

        I tried to enable following two in cluster.yaml

        hadoop::common_hdfs::hadoop_http_authentication_type: "%{hiera('hadoop::hadoop_security_authentication')}"
        hadoop::common_hdfs::generate_secrets: true
        

        But failed at this line:

        Error: Unknown function trocla at /bigtop-home/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:235 on node bigtop1.docker
        

        Sorry, I'm not quite sure how this is going to work.
        Would you please add some words in our puppet's README.md to document the feature you brings in?
        It would be great to instruct users which variables to enable for a particular feature to take place

        Show
        evans_ye Evans Ye added a comment - I tried to enable following two in cluster.yaml hadoop::common_hdfs::hadoop_http_authentication_type: "%{hiera('hadoop::hadoop_security_authentication')}" hadoop::common_hdfs::generate_secrets: true But failed at this line: Error: Unknown function trocla at /bigtop-home/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:235 on node bigtop1.docker Sorry, I'm not quite sure how this is going to work. Would you please add some words in our puppet's README.md to document the feature you brings in? It would be great to instruct users which variables to enable for a particular feature to take place
        Hide
        michaelweiser Michael Weiser added a comment - - edited

        Hi Evans Ye,

        I totally agree on the documentation front but hadn't thought as far because I wanted to get a feel for your acceptance of the idea first. So let's try if it works for you and then decide. Good to known that at least it doesn't cause any failures until enabled.

        The error you're getting is because trocla isn't installed yet. Unfortunately, trocla installation using the trocla puppet module seems to be a bit more broken than I initially thought. I'm working with the author on this and I am creating Debian packages for it. If you have Internet connectivity, you can run the following:

        # gem install trocla
        # puppet module install duritong/trocla
        # puppet apply -e "class { 'trocla::config': manage_dependencies => false }"
        

        The trocla ruby gem pulls in highline, moneta and bcrypt. The bcrypt gem might need ruby development packages (ruby.h) and a compiler. Alternatively you can use your distributions' packages. However you install, the following should now work as should the cookie generating in hadoop::common_hdfs:

        # puppet apply -e "file { '/tmp/blub': content => trocla("test", "plain") }"
        Notice: Compiled catalog for btnode1.proto.bsi.de in environment production in 0.43 seconds
        Notice: /Stage[main]/Main/File[/tmp/blub]/ensure: defined content as '{md5}505009853e199761c392f9fea8110648'
        Notice: Finished catalog run in 0.05 seconds
        # cat /tmp/blub
        puGNOX-G%zYDKHet
        

        With just puppet apply you won't get the full experience since password storage will happen in /tmp/trocla.yaml locally on each node. So, passwords will differ between nodes. In a master/agent setup, you'll get one trocla.yaml in /var/lib/puppet/server_storage on the master and passwords will be identical across hosts as needed for the cookie secret.

        Show
        michaelweiser Michael Weiser added a comment - - edited Hi Evans Ye , I totally agree on the documentation front but hadn't thought as far because I wanted to get a feel for your acceptance of the idea first. So let's try if it works for you and then decide. Good to known that at least it doesn't cause any failures until enabled. The error you're getting is because trocla isn't installed yet. Unfortunately, trocla installation using the trocla puppet module seems to be a bit more broken than I initially thought. I'm working with the author on this and I am creating Debian packages for it. If you have Internet connectivity, you can run the following: # gem install trocla # puppet module install duritong/trocla # puppet apply -e "class { 'trocla::config': manage_dependencies => false }" The trocla ruby gem pulls in highline, moneta and bcrypt. The bcrypt gem might need ruby development packages (ruby.h) and a compiler. Alternatively you can use your distributions' packages. However you install, the following should now work as should the cookie generating in hadoop::common_hdfs: # puppet apply -e "file { '/tmp/blub': content => trocla("test", "plain") }" Notice: Compiled catalog for btnode1.proto.bsi.de in environment production in 0.43 seconds Notice: /Stage[main]/Main/File[/tmp/blub]/ensure: defined content as '{md5}505009853e199761c392f9fea8110648' Notice: Finished catalog run in 0.05 seconds # cat /tmp/blub puGNOX-G%zYDKHet With just puppet apply you won't get the full experience since password storage will happen in /tmp/trocla.yaml locally on each node. So, passwords will differ between nodes. In a master/agent setup, you'll get one trocla.yaml in /var/lib/puppet/server_storage on the master and passwords will be identical across hosts as needed for the cookie secret.
        Hide
        michaelweiser Michael Weiser added a comment -

        Updated patch that adds some documentation and avoids a warning in the pupper master log because of not properly quoted \. in regexp in regsubst.

        Show
        michaelweiser Michael Weiser added a comment - Updated patch that adds some documentation and avoids a warning in the pupper master log because of not properly quoted \. in regexp in regsubst.
        Hide
        michaelweiser Michael Weiser added a comment -

        Updated patch that does not refresh the services upon keytab change because our host_keytab type will cause them to be restartet on every Puppet run. Also, keytab change shouldn't normally require a service restart.

        Show
        michaelweiser Michael Weiser added a comment - Updated patch that does not refresh the services upon keytab change because our host_keytab type will cause them to be restartet on every Puppet run. Also, keytab change shouldn't normally require a service restart.
        Hide
        evans_ye Evans Ye added a comment -

        It's a little bit difficult for me to install trocla on centos, the ruby version for centos is below 1.9, and even I manually upgrade to 1.9.3, the gem install trocla failed. I'll try this on debian instead.

        Show
        evans_ye Evans Ye added a comment - It's a little bit difficult for me to install trocla on centos, the ruby version for centos is below 1.9, and even I manually upgrade to 1.9.3, the gem install trocla failed. I'll try this on debian instead.
        Hide
        evans_ye Evans Ye added a comment -

        I think I have make this work on my debian testing cluster. But how can I login to those web UIs? I don't know where the username and auto generated password stored.
        In addition, I found ruby-dev is required by gem install trolca. So we better document this down.
        It would be great to have trolca installation implemented in bigtop_toolchain, [~Michael Weiser] could you please create another jira to hande this?

        Show
        evans_ye Evans Ye added a comment - I think I have make this work on my debian testing cluster. But how can I login to those web UIs? I don't know where the username and auto generated password stored. In addition, I found ruby-dev is required by gem install trolca . So we better document this down. It would be great to have trolca installation implemented in bigtop_toolchain, [~Michael Weiser] could you please create another jira to hande this?
        Hide
        oflebbe Olaf Flebbe added a comment -

        Login works like every SPNEGO enabled Web application:

        On your client machine run "kinit username" to fetch a kerberos ticket. Verify ticket with "klist". Start browser and enable SPNEGO with host: for firefox enter about:config and set network.negotiate-auth.trusted-uris to "http://", browse to app.

        Show
        oflebbe Olaf Flebbe added a comment - Login works like every SPNEGO enabled Web application: On your client machine run "kinit username" to fetch a kerberos ticket. Verify ticket with "klist". Start browser and enable SPNEGO with host: for firefox enter about:config and set network.negotiate-auth.trusted-uris to "http://", browse to app.
        Hide
        oflebbe Olaf Flebbe added a comment -

        Why implement installing ruby-dev with bigtop_toolchain? I learned in BIGTOP-1642 it is only intended for compile time dependencies.

        Show
        oflebbe Olaf Flebbe added a comment - Why implement installing ruby-dev with bigtop_toolchain? I learned in BIGTOP-1642 it is only intended for compile time dependencies.
        Hide
        cos Konstantin Boudnik added a comment -

        Is ruby_dev something that needs to be there for the runtime? As in "on the cluster"? That's what I am gathering from the README. If so, shall the deployment recipe be alternated?

        Show
        cos Konstantin Boudnik added a comment - Is ruby_dev something that needs to be there for the runtime? As in "on the cluster"? That's what I am gathering from the README. If so, shall the deployment recipe be alternated?
        Hide
        michaelweiser Michael Weiser added a comment -

        Konstantin Boudnik: The bcrypt gem seems to contain some C source that needs to be compiled upon installation. After that it's not needed at runtime. If bcrypt is installed as a precompiled system package (e.g. ruby-bcrypt on Debian and seemlingly bcrypt-ruby on Fedora/RedHat), then ruby-dev isn't needed either or if it were, it'd be pulled in automatically by the package manager.

        Also, bcrypt is only used by trocla. Since usage of trocla for the cookie secret only makes sense in a puppet master/agent setup (with puppet apply passwords will differ between machiens), bcrypt is only needed on the puppet master, not on all cluster nodes.

        Show
        michaelweiser Michael Weiser added a comment - Konstantin Boudnik : The bcrypt gem seems to contain some C source that needs to be compiled upon installation. After that it's not needed at runtime. If bcrypt is installed as a precompiled system package (e.g. ruby-bcrypt on Debian and seemlingly bcrypt-ruby on Fedora/RedHat), then ruby-dev isn't needed either or if it were, it'd be pulled in automatically by the package manager. Also, bcrypt is only used by trocla. Since usage of trocla for the cookie secret only makes sense in a puppet master/agent setup (with puppet apply passwords will differ between machiens), bcrypt is only needed on the puppet master, not on all cluster nodes.
        Hide
        cos Konstantin Boudnik added a comment -

        I am confused a little bit: our deployment modus operandi is master-less. Will this approach still work. Also, this:

        bcrypt gem seems to contain some C source that needs to be compiled upon installation

        That'd be great if we can avoid install-time-compiles as it complicates the environment dependencies and might not work in some cases. E.g. having SDK - JDK or C comp. - isn't a requirement for Hadoop cluster. Demanding to have a dev. env. for the installation process along seems like an extra burden that I am not sure we have to carry. Thoughts?

        Show
        cos Konstantin Boudnik added a comment - I am confused a little bit: our deployment modus operandi is master-less. Will this approach still work. Also, this: bcrypt gem seems to contain some C source that needs to be compiled upon installation That'd be great if we can avoid install-time-compiles as it complicates the environment dependencies and might not work in some cases. E.g. having SDK - JDK or C comp. - isn't a requirement for Hadoop cluster. Demanding to have a dev. env. for the installation process along seems like an extra burden that I am not sure we have to carry. Thoughts?
        Hide
        michaelweiser Michael Weiser added a comment -

        That's exactly the reason why I made the use of trocla an optional functionality that is disabled by default. Only after hadoop::common_hdfs::generate_secrets has been explicitly set to true all of this is necessary. If not, the secret can and must be supplied using hadoop::common_hdfs::hadoop_http_authentication_signature_secret as with all the other secrets. We could even have an insecure default value in the code for it, if we wanted (I wouldn't). So puppet apply will still work if you provide the password e.g. via hiera. Should I try to more clearly reflect that default behaviour in the README?

        Also, I am actively trying to get trocla and puppet-module-trocla packages into Debian. They already have all the dependencies as binary packages so that the whole installation process comes down to "apt-get install puppet-module-duritong-trocla".

        Show
        michaelweiser Michael Weiser added a comment - That's exactly the reason why I made the use of trocla an optional functionality that is disabled by default. Only after hadoop::common_hdfs::generate_secrets has been explicitly set to true all of this is necessary. If not, the secret can and must be supplied using hadoop::common_hdfs::hadoop_http_authentication_signature_secret as with all the other secrets. We could even have an insecure default value in the code for it, if we wanted (I wouldn't). So puppet apply will still work if you provide the password e.g. via hiera. Should I try to more clearly reflect that default behaviour in the README? Also, I am actively trying to get trocla and puppet-module-trocla packages into Debian. They already have all the dependencies as binary packages so that the whole installation process comes down to "apt-get install puppet-module-duritong-trocla".
        Hide
        cos Konstantin Boudnik added a comment -

        Thanks for the detailed clarification, Michael! Appreciate that.
        I think it'd be good to add a more detailed explanation to the README, so there's no confusion about the expectations, etc.

        Show
        cos Konstantin Boudnik added a comment - Thanks for the detailed clarification, Michael! Appreciate that. I think it'd be good to add a more detailed explanation to the README, so there's no confusion about the expectations, etc.
        Hide
        michaelweiser Michael Weiser added a comment -

        Here's the updated patch where I tried to clarify the instructions and also extended optional trocla usage to httpfs.

        Show
        michaelweiser Michael Weiser added a comment - Here's the updated patch where I tried to clarify the instructions and also extended optional trocla usage to httpfs.
        Hide
        cos Konstantin Boudnik added a comment -

        Can anyone review it so we can finish it of? Thanks!

        Show
        cos Konstantin Boudnik added a comment - Can anyone review it so we can finish it of? Thanks!
        Hide
        oflebbe Olaf Flebbe added a comment - - edited

        I did a review . +1 . Will commit it

        Oops, does not apply at README.md. Need a rebased patch

        Show
        oflebbe Olaf Flebbe added a comment - - edited I did a review . +1 . Will commit it Oops, does not apply at README.md. Need a rebased patch
        Hide
        michaelweiser Michael Weiser added a comment -

        Here's the rebased patch. It includes some very minor improvements:

        • fail if secret is undef
        • remove useless use of inline_template() in generation of secrets files
        Show
        michaelweiser Michael Weiser added a comment - Here's the rebased patch. It includes some very minor improvements: fail if secret is undef remove useless use of inline_template() in generation of secrets files
        Hide
        oflebbe Olaf Flebbe added a comment -

        LGTM, will commit

        Show
        oflebbe Olaf Flebbe added a comment - LGTM, will commit
        Hide
        cos Konstantin Boudnik added a comment -

        Olaf Flebbe, do you need help with committing it?

        Show
        cos Konstantin Boudnik added a comment - Olaf Flebbe , do you need help with committing it?
        Hide
        oflebbe Olaf Flebbe added a comment - - edited

        Ooops, forgot to close ticket.

        Show
        oflebbe Olaf Flebbe added a comment - - edited Ooops, forgot to close ticket.

          People

          • Assignee:
            michaelweiser Michael Weiser
            Reporter:
            michaelweiser Michael Weiser
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development