Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 0.9.0
    • None

    Description

      Now that Bigtop is publishing artifacts (RPMs, debian packages) we should use those in Whirr. There are also Puppet recipes, which would simplify things for Whirr.

      Attachments

        Activity

          tomwhite Thomas White added a comment -

          Attaching a work-in-progress patch that I did a little while ago. This uses the Bigtop Puppet scripts to start HDFS, YARN, and ZooKeeper (just a single node).

          Some things that are missing (could be follow-on JIRAs):

          • Integration tests
          • Better namespacing of roles. At the moment you say 'puppet:hadoop::namenode' which is not very pretty, but perhaps we could just overload 'hadoop-namenode' and have a role-namespace property or similar.
          • Improvement in registering subroles in PuppetClusterActionHandlerFactory.
          • Ubuntu support (needs changes in Bigtop perhaps).

          I'm not working on this at the moment so I'd be happy for someone to take it up if interested.

          tomwhite Thomas White added a comment - Attaching a work-in-progress patch that I did a little while ago. This uses the Bigtop Puppet scripts to start HDFS, YARN, and ZooKeeper (just a single node). Some things that are missing (could be follow-on JIRAs): Integration tests Better namespacing of roles. At the moment you say 'puppet:hadoop::namenode' which is not very pretty, but perhaps we could just overload 'hadoop-namenode' and have a role-namespace property or similar. Improvement in registering subroles in PuppetClusterActionHandlerFactory. Ubuntu support (needs changes in Bigtop perhaps). I'm not working on this at the moment so I'd be happy for someone to take it up if interested.
          tomwhite Thomas White added a comment -

          Moved attachment to WHIRR-516 sub task.

          tomwhite Thomas White added a comment - Moved attachment to WHIRR-516 sub task.
          stevel@apache.org Steve Loughran added a comment -
          1. do the puppet scripts have to be used to start a bigtop-installation? That is: could this service be a template to install the RPMs/debs that can then be started by other means (including the classic service hadoop-namenode start)
          2. structuring this to be RPM/deb only could clean up the existing state of the installers, which are often tarball-first.
          stevel@apache.org Steve Loughran added a comment - do the puppet scripts have to be used to start a bigtop-installation? That is: could this service be a template to install the RPMs/debs that can then be started by other means (including the classic service hadoop-namenode start ) structuring this to be RPM/deb only could clean up the existing state of the installers, which are often tarball-first.
          tomwhite Thomas White added a comment -

          1. do the puppet scripts have to be used to start a bigtop-installation?

          No. That's what WHIRR-676 is for - just using the RPMs/debs to do the installation and do service start as you say. I think we should do this first as it is a minor change. The puppet stuff can be done later (WHIRR-516).

          tomwhite Thomas White added a comment - 1. do the puppet scripts have to be used to start a bigtop-installation? No. That's what WHIRR-676 is for - just using the RPMs/debs to do the installation and do service start as you say. I think we should do this first as it is a minor change. The puppet stuff can be done later ( WHIRR-516 ).

          People

            Unassigned Unassigned
            tomwhite Thomas White
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: