Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1958

Upgrade default repositories and docker images to 1.0

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 1.0.0, 1.1.0
    • Component/s: deployment
    • Labels:
      None

      Description

      The default repositories in puppet recipes and bigtop provisioner configurations are still 0.8.0. We should upgrade them to 1.0.0 repo and switch the docker image to 1.0 version as well.

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          The CNAME got created and is pointing to the current CI master hostname. The change will be propagated within an hour or so.

          We can change it as we migrate to the new infrastructure.

          Show
          cos Konstantin Boudnik added a comment - The CNAME got created and is pointing to the current CI master hostname. The change will be propagated within an hour or so. We can change it as we migrate to the new infrastructure.
          Hide
          cos Konstantin Boudnik added a comment -

          Here's the INFRA ticket.

          Show
          cos Konstantin Boudnik added a comment - Here's the INFRA ticket.
          Hide
          cos Konstantin Boudnik added a comment -

          I sent an email to the dev@community to figure out what'd be the right way of setting this up. Once we have the CNAME (one way or another) I will open a new ticket to update the URLs on master.

          Show
          cos Konstantin Boudnik added a comment - I sent an email to the dev@community to figure out what'd be the right way of setting this up. Once we have the CNAME (one way or another) I will open a new ticket to update the URLs on master.
          Hide
          evans_ye Evans Ye added a comment -

          Committed and pushed to master and branch 1.0. Thank you Konstantin Boudnik for a quick +1.

          Show
          evans_ye Evans Ye added a comment - Committed and pushed to master and branch 1.0. Thank you Konstantin Boudnik for a quick +1.
          Hide
          evans_ye Evans Ye added a comment -

          Make sense to me.
          To conclude for the others.
          I'll apply this patch to master and 1.0 branch and unlock the 1.0 release.
          The 1.0 release will still use a fixed url pointing to Cloudera Jenkins.
          Cos will help to file ticket to infra and setup a CNAME for the repos in next release.

          Show
          evans_ye Evans Ye added a comment - Make sense to me. To conclude for the others. I'll apply this patch to master and 1.0 branch and unlock the 1.0 release. The 1.0 release will still use a fixed url pointing to Cloudera Jenkins. Cos will help to file ticket to infra and setup a CNAME for the repos in next release.
          Hide
          cos Konstantin Boudnik added a comment -

          I guess it makes sense, thank you. Looks like we are using old "cloudera...." URL for the new instances right? If so - let's file INFRA ticket to get us a CNAME for ci.bigtop.apache.org or something similar so we can hide the implementation details of where the CI master. This way we'll be able to move the server without changing anything in the later time. Does it make sense?

          Please commit this and I will open the INFRA ticket if there's no objections. Thank you very much!

          Show
          cos Konstantin Boudnik added a comment - I guess it makes sense, thank you. Looks like we are using old "cloudera...." URL for the new instances right? If so - let's file INFRA ticket to get us a CNAME for ci.bigtop.apache.org or something similar so we can hide the implementation details of where the CI master. This way we'll be able to move the server without changing anything in the later time. Does it make sense? Please commit this and I will open the INFRA ticket if there's no objections. Thank you very much!
          Hide
          evans_ye Evans Ye added a comment -

          Good point Konstantin Boudnik. That was intended. The reason behind this is because our EC2 instances for release builds do not have enough local storage, hence I can only set the preserved artefacts number to 1 on Jenkins. If we set the url to a specific build number and click on the build button again, then the fixed number artefacts will all gone.
          In addition, if we have some fixes for 1.0 RC and need to roll out new binaries, using lastSuccessfulBuild pointer avoid code changes.
          If you're ok with my explanation, I'd like to commit this to master and cherry pick to 1.0. Is it ok to do so?

          Show
          evans_ye Evans Ye added a comment - Good point Konstantin Boudnik . That was intended. The reason behind this is because our EC2 instances for release builds do not have enough local storage, hence I can only set the preserved artefacts number to 1 on Jenkins. If we set the url to a specific build number and click on the build button again, then the fixed number artefacts will all gone. In addition, if we have some fixes for 1.0 RC and need to roll out new binaries, using lastSuccessfulBuild pointer avoid code changes. If you're ok with my explanation, I'd like to commit this to master and cherry pick to 1.0. Is it ok to do so?
          Hide
          cos Konstantin Boudnik added a comment -

          It's all good except for one thing: new repo URLs are pointing to the lastSuccessfulBuild which might be changed at any moment. Don't you think it might be an issue? Shall it be fixed?

          Other than that +1

          Show
          cos Konstantin Boudnik added a comment - It's all good except for one thing: new repo URLs are pointing to the lastSuccessfulBuild which might be changed at any moment. Don't you think it might be an issue? Shall it be fixed? Other than that +1
          Hide
          evans_ye Evans Ye added a comment - - edited

          What're in the patch:

          • Update default yum and apt repos from 0.8 to 1.0 in puppet site.pp
          • Update default yum and apt repos from 0.8 to 1.0 in vagrant-puppet-docker and vagrant-puppet-vm
          • Update default memory size for docker container from 2048 to 4096 in vagrant-puppet-docker/vagrantconfig.yaml since 2G ram fail the smoke-test running in container.
          • Update docker images to bigtop/deploy:debian-8 and bigtop/deploy:centos-6. User can download them from docker hub directly.

          I've ran deployment tests on both Docker and VM environment. Leveraging new rpms took me around 3~4mins to finish one node deployment, which is really fast.

          Show
          evans_ye Evans Ye added a comment - - edited What're in the patch: Update default yum and apt repos from 0.8 to 1.0 in puppet site.pp Update default yum and apt repos from 0.8 to 1.0 in vagrant-puppet-docker and vagrant-puppet-vm Update default memory size for docker container from 2048 to 4096 in vagrant-puppet-docker/vagrantconfig.yaml since 2G ram fail the smoke-test running in container. Update docker images to bigtop/deploy:debian-8 and bigtop/deploy:centos-6. User can download them from docker hub directly. I've ran deployment tests on both Docker and VM environment. Leveraging new rpms took me around 3~4mins to finish one node deployment, which is really fast.

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              evans_ye Evans Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development