Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1564

docker-puppet doesn't use vagrantconfg.yaml for configuration parameters

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0
    • Component/s: deployment
    • Labels:
      None

      Description

      Looks like provision.sh has hardcoded content for puppet site.csv configuration file, instead of getting needed parameters from the YAML config.

        Issue Links

          Activity

          Hide
          evans_ye Evans Ye added a comment -

          Committed.
          I tested the feature again and it works well.
          The next step for the improvement is to combine docker and vagrant yaml to a single configuration file.

          Show
          evans_ye Evans Ye added a comment - Committed. I tested the feature again and it works well. The next step for the improvement is to combine docker and vagrant yaml to a single configuration file.
          Hide
          evans_ye Evans Ye added a comment -

          Konstantin Boudnik, thanks for the review, I'll apply this patch again for a test.
          If it works, I'll commit this directly

          Show
          evans_ye Evans Ye added a comment - Konstantin Boudnik , thanks for the review, I'll apply this patch again for a test. If it works, I'll commit this directly
          Hide
          cos Konstantin Boudnik added a comment -

          without trying this - it looks ok. So if it works as expected - +1, please commit.

          Show
          cos Konstantin Boudnik added a comment - without trying this - it looks ok. So if it works as expected - +1, please commit.
          Hide
          evans_ye Evans Ye added a comment -

          Uploaded a patch that gets configurations from yaml file.
          What the patch do is to extract configs from vagrantconfig.yaml and then fill them in to provision.sh as arguments. The entire process is implemented in docker-hadoop.sh.

          Show
          evans_ye Evans Ye added a comment - Uploaded a patch that gets configurations from yaml file. What the patch do is to extract configs from vagrantconfig.yaml and then fill them in to provision.sh as arguments. The entire process is implemented in docker-hadoop.sh .
          Hide
          cos Konstantin Boudnik added a comment -

          Does anyone has a spare cycle or two to update the toolchain (as a separate JIRA to avoid blocking this one)? It's gonna be nuts a couple of days for me, but after that I should be able to look at it as well.

          Show
          cos Konstantin Boudnik added a comment - Does anyone has a spare cycle or two to update the toolchain (as a separate JIRA to avoid blocking this one)? It's gonna be nuts a couple of days for me, but after that I should be able to look at it as well.
          Hide
          evans_ye Evans Ye added a comment -

          Hey folks, a conservative move is to limit Vagrant version in Vagrantfile, just add a line:

          Vagrant.require_version ">= 1.6.0"
          

          In addition, automatically installation/upgrade provided in bigtop_toolchain seems a good way to go!

          Show
          evans_ye Evans Ye added a comment - Hey folks, a conservative move is to limit Vagrant version in Vagrantfile, just add a line: Vagrant.require_version ">= 1.6.0" In addition, automatically installation/upgrade provided in bigtop_toolchain seems a good way to go!
          Hide
          jayunit100 jay vyas added a comment - - edited

          Yup i guess so... I think we can easily just link to vagrant downloads page above and grab rpm or deb autodetecting the linux distro ... feel free to make a jira And we can look at it !

          Maybe this can be part of Romans ci effort?

          After all once in toolchain, now that we can install rpms from bigtop/output,
          We just put that in jenckins , and voila! A bigtop build rpm script + Docker-puppet which runs out smoke tests == New bigtop ci!

          Show
          jayunit100 jay vyas added a comment - - edited Yup i guess so... I think we can easily just link to vagrant downloads page above and grab rpm or deb autodetecting the linux distro ... feel free to make a jira And we can look at it ! Maybe this can be part of Romans ci effort? After all once in toolchain, now that we can install rpms from bigtop/output, We just put that in jenckins , and voila! A bigtop build rpm script + Docker-puppet which runs out smoke tests == New bigtop ci!
          Hide
          cos Konstantin Boudnik added a comment -

          . ill think if there is an efficient way to specify it there

          Normally, you can just specify the minimal version of a package, but that'd require a package to be in the repo somewhere. I guess we can do something similar to what's being done for JDK: download the package and install it if the local version is lesser than expected, can't we?

          Show
          cos Konstantin Boudnik added a comment - . ill think if there is an efficient way to specify it there Normally, you can just specify the minimal version of a package, but that'd require a package to be in the repo somewhere. I guess we can do something similar to what's being done for JDK: download the package and install it if the local version is lesser than expected, can't we?
          Hide
          jayunit100 jay vyas added a comment -

          sure, I'm okay with that idea. ill think if there is an efficient way to specify it there

          Show
          jayunit100 jay vyas added a comment - sure, I'm okay with that idea. ill think if there is an efficient way to specify it there
          Hide
          cos Konstantin Boudnik added a comment -

          Now I am thinking if it'd make sense to enforce expected version through toolchain?

          Show
          cos Konstantin Boudnik added a comment - Now I am thinking if it'd make sense to enforce expected version through toolchain?
          Hide
          jayunit100 jay vyas added a comment -

          Hi cos.

          Unfortunately, You gotta get the up to date one, I think, in general vagrant
          Is releasing updates pretty regular.

          The good news it's super easy ... Just grab :

          https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.5_x86_64.deb

          You'll find it generally works good. Vagrant bundles all its own ruby deps, so
          Having the default fedora/Debian package isn't super important .

          Show
          jayunit100 jay vyas added a comment - Hi cos. Unfortunately, You gotta get the up to date one, I think, in general vagrant Is releasing updates pretty regular. The good news it's super easy ... Just grab : https://dl.bintray.com/mitchellh/vagrant/vagrant_1.6.5_x86_64.deb You'll find it generally works good. Vagrant bundles all its own ruby deps, so Having the default fedora/Debian package isn't super important .
          Hide
          cos Konstantin Boudnik added a comment -

          BTW, guys - it seems that docker-puppet requires Vagrant 1.6, where's even latest Ubuntu comes with 1.43.
          Any chance this deployment can work with an earlier version of Vagrant?

          Show
          cos Konstantin Boudnik added a comment - BTW, guys - it seems that docker-puppet requires Vagrant 1.6, where's even latest Ubuntu comes with 1.43. Any chance this deployment can work with an earlier version of Vagrant?
          Hide
          evans_ye Evans Ye added a comment -

          Yeah, I'm working on this, trying to get yaml parameters in the shell script. I'll tackle it down in this jira. Thanks for bringing this up.

          Show
          evans_ye Evans Ye added a comment - Yeah, I'm working on this, trying to get yaml parameters in the shell script. I'll tackle it down in this jira. Thanks for bringing this up.
          Hide
          jayunit100 jay vyas added a comment -

          ping Evans Ye i think we discussed consolidating the vagrantconfig.yaml into one file so that this will be solved.

          Show
          jayunit100 jay vyas added a comment - ping Evans Ye i think we discussed consolidating the vagrantconfig.yaml into one file so that this will be solved.

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              cos Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development