Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.3.0
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      Along the lines of WHIRR-49, implement support for Puppet to bootstrap servers.

      1. WHIRR-255.patch
        27 kB
        Andrei Savu
      2. WHIRR-255+311.patch
        55 kB
        Andrei Savu
      3. WHIRR-255+311.patch
        54 kB
        Andrei Savu
      4. WHIRR-255.patch
        25 kB
        Andrei Savu
      5. WHIRR-255.patch
        19 kB
        Andrei Savu
      6. WHIRR-255.patch
        5 kB
        Andrei Savu

        Issue Links

          Activity

          Hide
          Eugene Koontz added a comment -

          This might be useful: https://github.com/hstack/puppet

          Show
          Eugene Koontz added a comment - This might be useful: https://github.com/hstack/puppet
          Hide
          Tom White added a comment -

          Using Puppet in standalone mode might be a good way to start this, then there is no need to have a Puppet master running (though we might want to add one later).

          The install script would be something like:

          function install_puppet() {
            if which rpm &> /dev/null; then
              yum install -y ruby-devel ruby-irb ruby-rdoc rubygems
          
              wget http://puppetlabs.com/downloads/facter/facter-latest.tgz
              tar xf facter-latest.tgz
              cd facter-*; ruby install.rb; cd -
          
              wget http://puppetlabs.com/downloads/puppet/puppet-latest.tgz
              tar xf puppet-latest.tgz
              cd puppet-*; sudo ruby install.rb; cd -
            fi
          }
          

          Then instead of bash scripts a service could use puppet scripts (copied to the nodes just like the Hadoop service copies configuration files), running them with puppet apply. Could this work?

          Show
          Tom White added a comment - Using Puppet in standalone mode might be a good way to start this, then there is no need to have a Puppet master running (though we might want to add one later). The install script would be something like: function install_puppet() { if which rpm &> /dev/ null ; then yum install -y ruby-devel ruby-irb ruby-rdoc rubygems wget http: //puppetlabs.com/downloads/facter/facter-latest.tgz tar xf facter-latest.tgz cd facter-*; ruby install.rb; cd - wget http: //puppetlabs.com/downloads/puppet/puppet-latest.tgz tar xf puppet-latest.tgz cd puppet-*; sudo ruby install.rb; cd - fi } Then instead of bash scripts a service could use puppet scripts (copied to the nodes just like the Hadoop service copies configuration files), running them with puppet apply . Could this work?
          Hide
          James Turnbull added a comment -

          Easier might be:

          function install_puppet() {
            if which rpm &> /dev/null; then
              yum install -y ruby-devel ruby-irb ruby-rdoc rubygems
          
              gem install facter puppet
            fi
          }
          
          Show
          James Turnbull added a comment - Easier might be: function install_puppet() { if which rpm &> /dev/ null ; then yum install -y ruby-devel ruby-irb ruby-rdoc rubygems gem install facter puppet fi }
          Hide
          Andrei Savu added a comment -

          I just had a chat with the guys behind hstack.org and they would love to help us reuse the Puppet recipes they wrote. I will work side by side with them for the following 3 months.

          Show
          Andrei Savu added a comment - I just had a chat with the guys behind hstack.org and they would love to help us reuse the Puppet recipes they wrote. I will work side by side with them for the following 3 months.
          Hide
          Tom White added a comment -

          Great!

          Show
          Tom White added a comment - Great!
          Hide
          James Turnbull added a comment -

          Andrei et al - let me know if you need any help - I work at Puppet Labs and am happy to devote some time in getting this to work.

          Show
          James Turnbull added a comment - Andrei et al - let me know if you need any help - I work at Puppet Labs and am happy to devote some time in getting this to work.
          Hide
          James Turnbull added a comment -

          This may work

          function install_puppet() {
            if which rpm &> /dev/null; then
              yum install -y ruby ruby-rdoc rubygems
          
              gem install facter puppet
            fi
          }
          
          Show
          James Turnbull added a comment - This may work function install_puppet() { if which rpm &> /dev/ null ; then yum install -y ruby ruby-rdoc rubygems gem install facter puppet fi }
          Hide
          Andrei Savu added a comment -

          Just the basics: install latest Puppet. I've created a new type of statement for Puppet scripts. Tested with virtualbox (ubuntu 10.10 and centos 5.5) and aws-ec2 (ubuntu 10.04 & amazon ami).

          Show
          Andrei Savu added a comment - Just the basics: install latest Puppet. I've created a new type of statement for Puppet scripts. Tested with virtualbox (ubuntu 10.10 and centos 5.5) and aws-ec2 (ubuntu 10.04 & amazon ami).
          Hide
          Tom White added a comment -

          I don't think we should add the install_puppet call to ScriptBasedClusterAction - handlers should add it themselves if they use Puppet. (This is true of install_runurl now too, so we should remove it in a separate JIRA.)

          It would be good to have something to test the abstraction - perhaps add a new service, or rewrite an existing one (e.g. ZooKeeper)?

          Show
          Tom White added a comment - I don't think we should add the install_puppet call to ScriptBasedClusterAction - handlers should add it themselves if they use Puppet. (This is true of install_runurl now too, so we should remove it in a separate JIRA.) It would be good to have something to test the abstraction - perhaps add a new service, or rewrite an existing one (e.g. ZooKeeper)?
          Hide
          Andrei Savu added a comment -

          Indeed. I hope to be able to rewrite the ZooKeeper install / configure scripts by the end of the week.

          Show
          Andrei Savu added a comment - Indeed. I hope to be able to rewrite the ZooKeeper install / configure scripts by the end of the week.
          Hide
          Andrei Savu added a comment -

          I've done more work on this (created a new set of roles for puppet related components: puppet-master, puppet-daemon, puppet). How should we go about managing multiple configuration scripts for services? How should a configuration for a cluster that uses puppet ideally look like?

          Show
          Andrei Savu added a comment - I've done more work on this (created a new set of roles for puppet related components: puppet-master, puppet-daemon, puppet). How should we go about managing multiple configuration scripts for services? How should a configuration for a cluster that uses puppet ideally look like?
          Hide
          Andrei Savu added a comment -

          In this patch I've added Puppet as a service and performed some testing on EC2.

          Next I'm planning to use SPI to discover CLI commands so that we can add new commands in services (e.g. run-puppet).

          The patch is far from being ready but I appreciate any feedback.

          Show
          Andrei Savu added a comment - In this patch I've added Puppet as a service and performed some testing on EC2. Next I'm planning to use SPI to discover CLI commands so that we can add new commands in services (e.g. run-puppet). The patch is far from being ready but I appreciate any feedback.
          Hide
          Adrian Cole added a comment -

          andrei I think that the next step is to have alternate service configuration sets that use puppet manifests for configuration. puppet is more a supporting service to achieve a cluster configuration in this role. perhaps an easier one would be to start with cassandra: http://www.edwardcapriolo.com/roller/edwardcapriolo/entry/easy_street_deploying_cassandra_via

          Show
          Adrian Cole added a comment - andrei I think that the next step is to have alternate service configuration sets that use puppet manifests for configuration. puppet is more a supporting service to achieve a cluster configuration in this role. perhaps an easier one would be to start with cassandra: http://www.edwardcapriolo.com/roller/edwardcapriolo/entry/easy_street_deploying_cassandra_via
          Hide
          Andrei Savu added a comment -

          I need the ability to register a new command in the puppet service that's going to be used to run puppet on nodes.

          Show
          Andrei Savu added a comment - I need the ability to register a new command in the puppet service that's going to be used to run puppet on nodes.
          Hide
          Andrei Savu added a comment -

          In this patch I've added a new CLI command for triggering the execution of the puppet client on all the instances.

          One known issue: it's not possible to run both the master and the client on the same machine.

          Next: add git support for puppet-master. allow users to pull and push manifest files. by using this it should be easy to interact with running clusters.

          Next: create puppet recipe for deploying Cassandra (as Adrian suggested)

          Show
          Andrei Savu added a comment - In this patch I've added a new CLI command for triggering the execution of the puppet client on all the instances. One known issue: it's not possible to run both the master and the client on the same machine. Next: add git support for puppet-master. allow users to pull and push manifest files. by using this it should be easy to interact with running clusters. Next: create puppet recipe for deploying Cassandra (as Adrian suggested)
          Hide
          Andrei Savu added a comment -

          In this patch:

          • manage puppet manifests using git
          • run-puppet works for all nodes

          Next:

          • add support for multiple install / configure scripts for services
          • migrate zookeeper or cassandra

          I will get back to this in a few days. Now I'm working on adding the ability to repair a ZooKeeper cluster for a demo.

          Show
          Andrei Savu added a comment - In this patch: manage puppet manifests using git run-puppet works for all nodes Next: add support for multiple install / configure scripts for services migrate zookeeper or cassandra I will get back to this in a few days. Now I'm working on adding the ability to repair a ZooKeeper cluster for a demo.
          Hide
          Andrei Savu added a comment -

          Updated patch for the current trunk. I will move forward to get to the point where using Puppet scripts for a new service or for rewriting an existing one is a viable option.

          I guess the interface should be similar to the jclouds statements with scripts stored as resources.

          Show
          Andrei Savu added a comment - Updated patch for the current trunk. I will move forward to get to the point where using Puppet scripts for a new service or for rewriting an existing one is a viable option. I guess the interface should be similar to the jclouds statements with scripts stored as resources.
          Hide
          Chad Metcalf added a comment -

          I might suggest you follow a slightly different model. Take vagrant for example. There are two routes to configure a box with puppet on vagrant:

          Standalone, via puppet apply. Set a manifest/modules path and viola masterless install. http://vagrantup.com/docs/provisioners/puppet.html

          Server, via puppet agent. Set a puppet server and point the node to it. Standard puppet rules apply. http://vagrantup.com/docs/provisioners/puppet_server.html

          If you need the functionality associated with a full puppet master you'll likely want more control over its configuration then you're going to want to encode in a template. thin configs/pluginsync/stored config db/etc. If you don't need that level of functionality, masterless would be easier.

          Show
          Chad Metcalf added a comment - I might suggest you follow a slightly different model. Take vagrant for example. There are two routes to configure a box with puppet on vagrant: Standalone, via puppet apply. Set a manifest/modules path and viola masterless install. http://vagrantup.com/docs/provisioners/puppet.html Server, via puppet agent. Set a puppet server and point the node to it. Standard puppet rules apply. http://vagrantup.com/docs/provisioners/puppet_server.html If you need the functionality associated with a full puppet master you'll likely want more control over its configuration then you're going to want to encode in a template. thin configs/pluginsync/stored config db/etc. If you don't need that level of functionality, masterless would be easier.
          Hide
          Andrei Savu added a comment -

          Chad, thanks for feedback. I understand the trade-offs and I believe it's more useful to have a Puppet master that can be used as a central repository for cluster configuration (also accessible as a git repository).

          I'm thinking about Puppet support in Whirr as an alternative service configuration mechanism when creating a new cluster and at the same time as a mechanism that can be used to manage cluster later on (we are lacking something like this). Does it make sense for you?

          Show
          Andrei Savu added a comment - Chad, thanks for feedback. I understand the trade-offs and I believe it's more useful to have a Puppet master that can be used as a central repository for cluster configuration (also accessible as a git repository). I'm thinking about Puppet support in Whirr as an alternative service configuration mechanism when creating a new cluster and at the same time as a mechanism that can be used to manage cluster later on (we are lacking something like this). Does it make sense for you?
          Hide
          Chad Metcalf added a comment -

          It sure does. My concern is that abstracting a fully working master is a lot to bite off. For example we use exported resources very common especially for Hadoop clusters that would use monitoring (nagios) or security (kerberos). So now you've got to deploy a database.

          I agree 100% that puppet is great for a mechanism that can be used to manage a cluster not only at deployment but in the future. My point was to build start even more simple. Have the capability to use masterless or point to a master. Then once that is working if you've time/use for it tackle building a master from scratch.

          Also from my use case, I've already got a master in ec2 that I can point this to. The above allows for that as well as building your own.

          Show
          Chad Metcalf added a comment - It sure does. My concern is that abstracting a fully working master is a lot to bite off. For example we use exported resources very common especially for Hadoop clusters that would use monitoring (nagios) or security (kerberos). So now you've got to deploy a database. I agree 100% that puppet is great for a mechanism that can be used to manage a cluster not only at deployment but in the future. My point was to build start even more simple. Have the capability to use masterless or point to a master. Then once that is working if you've time/use for it tackle building a master from scratch. Also from my use case, I've already got a master in ec2 that I can point this to. The above allows for that as well as building your own.
          Hide
          Andrei Savu added a comment -

          I understand. I will keep that in mind and rework the patch to match your requirements.

          We could to this in a few steps, probably as multiple JIRAs:

          • masterless Puppet deployment (most of the components are already in place)
          • dummy Puppet master server (no database, no special configs, version control support) with Puppet agents executed as needed by running "whirr run-puppet" (this approach implemented by the current patch)
          • only Puppet agents using an external Puppet master - I believe we are going to need to have a convention in place for setting hostnames. Are IPs good enough? How should we handle certificates? Auto-signing?
          Show
          Andrei Savu added a comment - I understand. I will keep that in mind and rework the patch to match your requirements. We could to this in a few steps, probably as multiple JIRAs: masterless Puppet deployment (most of the components are already in place) dummy Puppet master server (no database, no special configs, version control support) with Puppet agents executed as needed by running "whirr run-puppet" (this approach implemented by the current patch) only Puppet agents using an external Puppet master - I believe we are going to need to have a convention in place for setting hostnames. Are IPs good enough? How should we handle certificates? Auto-signing?
          Hide
          Andrei Savu added a comment -

          I've found a nice article describing how Loggly uses a masterless Puppet setup:
          http://semicomplete.com/presentations/puppet-at-loggly/puppet-at-loggly.pdf.html

          What do you think?

          Show
          Andrei Savu added a comment - I've found a nice article describing how Loggly uses a masterless Puppet setup: http://semicomplete.com/presentations/puppet-at-loggly/puppet-at-loggly.pdf.html What do you think?
          Hide
          Chad Metcalf added a comment -

          I agree with your three steps above.

          As for the masterless puppet from Loggly. Jordan (aka Whack) is pretty much the most active masterless user I know. I think that we can start with the easy masterless. And if their is additional need/desire for the rest (exported resources and the like) we can add them.

          Between the three pathways above I think most puppet users would be well covered.

          What help do you need to make this happen?

          Show
          Chad Metcalf added a comment - I agree with your three steps above. As for the masterless puppet from Loggly. Jordan (aka Whack) is pretty much the most active masterless user I know. I think that we can start with the easy masterless. And if their is additional need/desire for the rest (exported resources and the like) we can add them. Between the three pathways above I think most puppet users would be well covered. What help do you need to make this happen?
          Hide
          Andrei Savu added a comment -

          > What help do you need to make this happen?

          Unfortunately I'm not going to be able to put enough time into this in the near future (weeks). For the following two weeks I'm going to work on an internal Adobe project and from July I will join Facebook in US.

          If you could takeover I would gladly help you with testing. Thanks Chad.

          Show
          Andrei Savu added a comment - > What help do you need to make this happen? Unfortunately I'm not going to be able to put enough time into this in the near future (weeks). For the following two weeks I'm going to work on an internal Adobe project and from July I will join Facebook in US. If you could takeover I would gladly help you with testing. Thanks Chad.
          Hide
          Adrian Cole added a comment -

          related issue in jclouds
          http://code.google.com/p/jclouds/issues/detail?id=603

          I'm going to puppet training next week so can help on this.

          Show
          Adrian Cole added a comment - related issue in jclouds http://code.google.com/p/jclouds/issues/detail?id=603 I'm going to puppet training next week so can help on this.
          Hide
          Adrian Cole added a comment -

          referring to http://semicomplete.com/presentations/puppet-at-loggly/puppet-at-loggly.pdf.html it seems nodeless setup would work well in whirr, as we could translate whirr roles into puppet role tags. thoughts?

          Show
          Adrian Cole added a comment - referring to http://semicomplete.com/presentations/puppet-at-loggly/puppet-at-loggly.pdf.html it seems nodeless setup would work well in whirr, as we could translate whirr roles into puppet role tags. thoughts?
          Hide
          Alex Heneveld added a comment -

          I'm interested in helping with some of this work. Seems sensible to start with the simpler masterless case. To avoid collisions with the patches on this issue, I've set up a new issue at WHIRR-385.

          Show
          Alex Heneveld added a comment - I'm interested in helping with some of this work. Seems sensible to start with the simpler masterless case. To avoid collisions with the patches on this issue, I've set up a new issue at WHIRR-385 .
          Hide
          Alex Heneveld added a comment -

          WHIRR-255 looks at masterFUL puppetry; WHIRR-385 at masterLESS puppet

          Show
          Alex Heneveld added a comment - WHIRR-255 looks at masterFUL puppetry; WHIRR-385 at masterLESS puppet
          Hide
          Adrian Cole added a comment -

          good idea, alex

          Show
          Adrian Cole added a comment - good idea, alex
          Hide
          Guido Serra aka Zeph added a comment -

          hi, was there any follow up on this? someone created WHIRR-681
          shall I create a new ticket specifically for the standard master driven topology?
          as I can see all that has been implemented is towards the masterless design

          Show
          Guido Serra aka Zeph added a comment - hi, was there any follow up on this? someone created WHIRR-681 shall I create a new ticket specifically for the standard master driven topology? as I can see all that has been implemented is towards the masterless design
          Hide
          Roman Shaposhnik added a comment -

          @Guido, WHIRR-681 would be me

          Now, as for the master-driven setup, could you elaborate what do you have in mind? I can see how signing CAs would be one of the things that this new service would have to handle, but what else?

          Show
          Roman Shaposhnik added a comment - @Guido, WHIRR-681 would be me Now, as for the master-driven setup, could you elaborate what do you have in mind? I can see how signing CAs would be one of the things that this new service would have to handle, but what else?
          Hide
          Guido Serra aka Zeph added a comment -

          Roman Shaposhnik I do need to have an environment in which each node is aware of the others

          classic web frontend/backend setup, with failover database and a couple of application servers

          to achieve that, I do need a master-driven setup...
          yes, the signing of the client certificates is one of the topics to be handled

          also, the "whirr run-puppet" to enforce the puppet agent run, would be a nice to have
          (u probably mentioned it in the other ticket)

          Show
          Guido Serra aka Zeph added a comment - Roman Shaposhnik I do need to have an environment in which each node is aware of the others classic web frontend/backend setup, with failover database and a couple of application servers to achieve that, I do need a master-driven setup... yes, the signing of the client certificates is one of the topics to be handled also, the "whirr run-puppet" to enforce the puppet agent run, would be a nice to have (u probably mentioned it in the other ticket)

            People

            • Assignee:
              Adrian Cole
              Reporter:
              Lars George
            • Votes:
              6 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:

                Development