Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: deployment
    • Labels:
      None

      Description

      As of BIGTOP-1908 , lets create a shell script which installs dependencies for bare-bone docker images and minimal OS installations.

      This will be reused for Dockerfiles (bigtop/puppet) and VM provisioning.

      1. BIGTOP-2018.1.patch
        9 kB
        Olaf Flebbe
      2. BIGTOP-2018.2.patch
        10 kB
        Olaf Flebbe

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          Makes sense! Sorry I missed the discussion somehow.

          Show
          cos Konstantin Boudnik added a comment - Makes sense! Sorry I missed the discussion somehow.
          Hide
          evans_ye Evans Ye added a comment - - edited

          In fact we've discussed this in BIGTOP-1996. Olaf Flebbe and I concluded to have the layout like this:

          bigtop/slaves:trunk-centos-6
          bigtop/slaves:1.0.0-centos-6
          

          We'll leave the scheme bigtop/slaves:distro as is and adopt new scheme for any new pushes.

          Show
          evans_ye Evans Ye added a comment - - edited In fact we've discussed this in BIGTOP-1996 . Olaf Flebbe and I concluded to have the layout like this: bigtop/slaves:trunk-centos-6 bigtop/slaves:1.0.0-centos-6 We'll leave the scheme bigtop/slaves:distro as is and adopt new scheme for any new pushes.
          Hide
          cos Konstantin Boudnik added a comment -

          Shall we tag it as bigtop/slaves:distro-1.x then? Or something along those lines?

          Show
          cos Konstantin Boudnik added a comment - Shall we tag it as bigtop/slaves:distro-1.x then? Or something along those lines?
          Hide
          oflebbe Olaf Flebbe added a comment -

          We cannot change bigtop/slaves:distro, since this would break 1.0 branch.

          Show
          oflebbe Olaf Flebbe added a comment - We cannot change bigtop/slaves:distro, since this would break 1.0 branch.
          Hide
          cos Konstantin Boudnik added a comment -

          Ah, that's right! Sorry for being dense, but it means that now "slaves" images would need to be regenerated and redeployed, right? Otherwise the changes won't get picked up?

          Show
          cos Konstantin Boudnik added a comment - Ah, that's right! Sorry for being dense, but it means that now "slaves" images would need to be regenerated and redeployed, right? Otherwise the changes won't get picked up?
          Hide
          evans_ye Evans Ye added a comment -

          The images I built and pushed are bigtop/puppet, which are base images for downstream bigtop/slaves and bigtop/deploy, as of our design in BIGTOP-1908.

          Show
          evans_ye Evans Ye added a comment - The images I built and pushed are bigtop/puppet , which are base images for downstream bigtop/slaves and bigtop/deploy, as of our design in BIGTOP-1908 .
          Hide
          cos Konstantin Boudnik added a comment -

          Sorry, which images are those? bigtop/slaves or else? Thanks!

          Show
          cos Konstantin Boudnik added a comment - Sorry, which images are those? bigtop/slaves or else? Thanks!
          Hide
          evans_ye Evans Ye added a comment - - edited

          Just for a record I've pushed newly built bigtop/puppet images based on this. The process I've done is to build images by this job, then go into that EC2 instance and push manaully.

          Show
          evans_ye Evans Ye added a comment - - edited Just for a record I've pushed newly built bigtop/puppet images based on this. The process I've done is to build images by this job , then go into that EC2 instance and push manaully.
          Hide
          evans_ye Evans Ye added a comment -

          +1 looks very nice to me. I've committed this so that other related task can kick off immediately.
          Thank you Olaf Flebbe.

          Show
          evans_ye Evans Ye added a comment - +1 looks very nice to me. I've committed this so that other related task can kick off immediately. Thank you Olaf Flebbe .
          Hide
          oflebbe Olaf Flebbe added a comment -

          It's late but I hope I fixed all the remaining issues

          Show
          oflebbe Olaf Flebbe added a comment - It's late but I hope I fixed all the remaining issues
          Hide
          cos Konstantin Boudnik added a comment -

          Yeah, absolutely. Not a blocker in a anyway. Thanks!

          Show
          cos Konstantin Boudnik added a comment - Yeah, absolutely. Not a blocker in a anyway. Thanks!
          Hide
          oflebbe Olaf Flebbe added a comment - - edited

          Evans Ye Thanks for your comments. Yes indeed I forgot about the hiera problem on ubuntu 14.04 .

          I will change the messages to "unsupported"

          Konstantin Boudnik Thanks for looking into the ticket too, Changing the name is rather tedious since I will have to change many places. Something like "make puppet work" seems to me more expressive than seeding a toolchain. In fact we (evans and myself) are proposing it to use it even as a first step to deploy on bare metal/vagrant/docker/openstack ... too, not installing the compile toolchain. Is it ok for you to leave it that way?

          Show
          oflebbe Olaf Flebbe added a comment - - edited Evans Ye Thanks for your comments. Yes indeed I forgot about the hiera problem on ubuntu 14.04 . I will change the messages to "unsupported" Konstantin Boudnik Thanks for looking into the ticket too, Changing the name is rather tedious since I will have to change many places. Something like "make puppet work" seems to me more expressive than seeding a toolchain. In fact we (evans and myself) are proposing it to use it even as a first step to deploy on bare metal/vagrant/docker/openstack ... too, not installing the compile toolchain. Is it ok for you to leave it that way?
          Hide
          cos Konstantin Boudnik added a comment -

          Good stuff! A minor comment: this looks like toolchain seeding script. Shall we call just that ie seeding.sh ?

          Show
          cos Konstantin Boudnik added a comment - Good stuff! A minor comment: this looks like toolchain seeding script. Shall we call just that ie seeding.sh ?
          Hide
          evans_ye Evans Ye added a comment -

          I find the code is very elegant to me and I learned a lot. Thanks for the patch!
          Here're three minor items I'd like to mention:

          • The puppetize.sh file is missing execution bit
          • There's an issue I discovered on Ubuntu(BIGTOP-2003), hence we need to install puppetlabs repo instead of using native OS repo.
          • I tried fedora 21 and got the message: Please add support for fedora_21 here.. Is it better to state that we're not supporting fedora_21? Since even though somebody add support for that OS, users can't build packages or deploy hadoop cluster on that OS anyway.
          Show
          evans_ye Evans Ye added a comment - I find the code is very elegant to me and I learned a lot. Thanks for the patch! Here're three minor items I'd like to mention: The puppetize.sh file is missing execution bit There's an issue I discovered on Ubuntu( BIGTOP-2003 ), hence we need to install puppetlabs repo instead of using native OS repo. I tried fedora 21 and got the message: Please add support for fedora_21 here. . Is it better to state that we're not supporting fedora_21? Since even though somebody add support for that OS, users can't build packages or deploy hadoop cluster on that OS anyway.

            People

            • Assignee:
              oflebbe Olaf Flebbe
              Reporter:
              oflebbe Olaf Flebbe
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development