Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2295

Docker tests should consume current builds

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.1
    • Component/s: build, ci
    • Labels:
      None

      Description

      Die Download URL of packages point to 1.0 bigtop builds. They should point to a current stable build.

        Issue Links

          Activity

          Hide
          evans_ye Evans Ye added a comment -

          Close as fixed.

          Show
          evans_ye Evans Ye added a comment - Close as fixed.
          Hide
          cos Konstantin Boudnik added a comment -

          Ah, I see - you're using the nightly repos. That's perhaps even better than the initial idea of using multiple repos, although a bit slower as the time is spent copying the artifacts around. On the flip side: the same nightly repos could be used by anyone who wants to do anything with nightlies.

          Thanks for fixing the matrix. I guess we can close this one now?

          Show
          cos Konstantin Boudnik added a comment - Ah, I see - you're using the nightly repos. That's perhaps even better than the initial idea of using multiple repos, although a bit slower as the time is spent copying the artifacts around. On the flip side: the same nightly repos could be used by anyone who wants to do anything with nightlies. Thanks for fixing the matrix. I guess we can close this one now?
          Hide
          oflebbe Olaf Flebbe added a comment -

          Evans Ye : Looks really good now! Thanks!

          Show
          oflebbe Olaf Flebbe added a comment - Evans Ye : Looks really good now! Thanks!
          Hide
          evans_ye Evans Ye added a comment -

          All works now. Thank you so much for getting the repo generation job fixed.
          The deployment matrix against up-to-date packages:
          https://ci.bigtop.apache.org/view/Provisioner/job/Bigtop-trunk-deployments/67/

          Show
          evans_ye Evans Ye added a comment - All works now. Thank you so much for getting the repo generation job fixed. The deployment matrix against up-to-date packages: https://ci.bigtop.apache.org/view/Provisioner/job/Bigtop-trunk-deployments/67/
          Hide
          cos Konstantin Boudnik added a comment -

          Yes, and we need to make sure that we set BIGTOP_BUILD_STAMP to the Jenkins build # to make sure the package version is monotonously growing. That will guarantee that we are installing the latest package if available rather than the base one from stable repo.

          Show
          cos Konstantin Boudnik added a comment - Yes, and we need to make sure that we set BIGTOP_BUILD_STAMP to the Jenkins build # to make sure the package version is monotonously growing. That will guarantee that we are installing the latest package if available rather than the base one from stable repo.
          Hide
          evans_ye Evans Ye added a comment - - edited

          I was trying to bring the deployment matrix back and move our experimental steps to https://ci.bigtop.apache.org/view/Provisioner/job/Bigtop-trunk-deployments-testing.
          Meanwhile, I saw you fix the build on June 15. Hence I've tried to set URLs to https://ci.bigtop.apache.org/job/Bigtop-trunk-repos/OS=${OS},label=docker-slave/ws/output.

          It seems that using a consolidated repo with 1.2.0 as fallback for failures is a good way to go.
          Are you thinking in the same way?

          Show
          evans_ye Evans Ye added a comment - - edited I was trying to bring the deployment matrix back and move our experimental steps to https://ci.bigtop.apache.org/view/Provisioner/job/Bigtop-trunk-deployments-testing . Meanwhile, I saw you fix the build on June 15. Hence I've tried to set URLs to https://ci.bigtop.apache.org/job/Bigtop-trunk-repos/OS=$ {OS},label=docker-slave/ws/output. It seems that using a consolidated repo with 1.2.0 as fallback for failures is a good way to go. Are you thinking in the same way?
          Hide
          cos Konstantin Boudnik added a comment -

          So, with all subtasks out of sight, shall we make the changes to the Jenkins deployment matrix to include a whole bunch of the repo URLs, using lastSuccessful package build pointers?

          Show
          cos Konstantin Boudnik added a comment - So, with all subtasks out of sight, shall we make the changes to the Jenkins deployment matrix to include a whole bunch of the repo URLs, using lastSuccessful package build pointers?
          Hide
          evans_ye Evans Ye added a comment -

          I'd like the former as well, it seems we have different naming across the project, which is quite confusing. We can fix the rest parts later.

          Show
          evans_ye Evans Ye added a comment - I'd like the former as well, it seems we have different naming across the project, which is quite confusing. We can fix the rest parts later.
          Hide
          cos Konstantin Boudnik added a comment - - edited

          Hmm, something weird is going on. I am looking into the configuration of Bigtop-trunk-deployment job and I see the following code among the build steps:

          if [ ${OS} == "centos6" ] || [ ${OS} == "centos7" ] || [ ${OS} == "fedora25" ]; then
            echo "Using repo from https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=${COMPONENTS},OS=${OS}/lastSuccessfulBuild/artifact/output"
            sed "s/repo.*/repo: https:\/\/ci.bigtop.apache.org\/view\/Packages\/job\/Bigtop-trunk-packages\/COMPONENTS=${COMPONENTS},OS=${OS}\/lastSuccessfulBuild\/artifact\/output/g" \
              provisioner/docker/${CONFIG} > provisioner/docker/${CONFIG}
          else
            echo "Using repo from https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=${COMPONENTS},OS=${OS}/lastSuccessfulBuild/artifact/output/apt"
            sed "s/repo.*/repo: https:\/\/ci.bigtop.apache.org\/view\/Packages\/job\/Bigtop-trunk-packages\/COMPONENTS=${COMPONENTS},OS=${OS}\/lastSuccessfulBuild\/artifact\/output\/apt/g" \
              provisioner/docker/${CONFIG} > provisioner/docker/${CONFIG}
          fi
          

          which indicates that we suppose to use the nightly packages. However, the code code is buggy because the OS names in this job configuration aren't matching OS names in the package build. E.g. centos6 vs centos-6; ubuntu-xenial vs ubuntu-16.04 and so on. As the result of the sed operation failure, we are using the whatever is set in the provisioner's configuration files.

          As I see it, we have two ways of fixing it:

          1. rename the provisioner's configs to use the same OS names as our CI matrices are using (centos-7, debian-8, ubuntu-16.04)
          2. fix the provisioner job to use the correct names in the repo URLs
            I really prefer the former fix, because it is more consistent, but i might break something else, possible in some user setup. I am not very concerned about that part, but it something to keep in mind.

          And one last thing: even when this bug is fixed, we won't be able to install/deploy most of the components, because of the failing dependencies. For that we need to figure out a way of specifying a list of repositories where different packages would be resolved from.

          Thoughts?

          Show
          cos Konstantin Boudnik added a comment - - edited Hmm, something weird is going on. I am looking into the configuration of Bigtop-trunk-deployment job and I see the following code among the build steps: if [ ${OS} == "centos6" ] || [ ${OS} == "centos7" ] || [ ${OS} == "fedora25" ]; then echo "Using repo from https: //ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=${COMPONENTS},OS=${OS}/lastSuccessfulBuild/artifact/output" sed "s/repo.*/repo: https:\/\/ci.bigtop.apache.org\/view\/Packages\/job\/Bigtop-trunk-packages\/COMPONENTS=${COMPONENTS},OS=${OS}\/lastSuccessfulBuild\/artifact\/output/g" \ provisioner/docker/${CONFIG} > provisioner/docker/${CONFIG} else echo "Using repo from https: //ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=${COMPONENTS},OS=${OS}/lastSuccessfulBuild/artifact/output/apt" sed "s/repo.*/repo: https:\/\/ci.bigtop.apache.org\/view\/Packages\/job\/Bigtop-trunk-packages\/COMPONENTS=${COMPONENTS},OS=${OS}\/lastSuccessfulBuild\/artifact\/output\/apt/g" \ provisioner/docker/${CONFIG} > provisioner/docker/${CONFIG} fi which indicates that we suppose to use the nightly packages. However, the code code is buggy because the OS names in this job configuration aren't matching OS names in the package build. E.g. centos6 vs centos-6 ; ubuntu-xenial vs ubuntu-16.04 and so on. As the result of the sed operation failure, we are using the whatever is set in the provisioner's configuration files. As I see it, we have two ways of fixing it: rename the provisioner's configs to use the same OS names as our CI matrices are using (centos-7, debian-8, ubuntu-16.04) fix the provisioner job to use the correct names in the repo URLs I really prefer the former fix, because it is more consistent, but i might break something else, possible in some user setup. I am not very concerned about that part, but it something to keep in mind. And one last thing: even when this bug is fixed, we won't be able to install/deploy most of the components, because of the failing dependencies. For that we need to figure out a way of specifying a list of repositories where different packages would be resolved from. Thoughts?
          Hide
          cos Konstantin Boudnik added a comment -

          Well, in the last of 1.2.1 to be coming soon, let's revive this and try to make it happen in the current release, shall we? Roman Shaposhnik, do you want to share your interesting ideas?

          What Olaf Flebbe has proposed would most likely work, although the patch needs to be re-worked slightly to reflect the fact that we aren't using Vagrant anymore. However, I am strongly in favor of doing exactly this because otherwise we don't have an official way of testing our packages until the release repo is ready, which is a far too late.

          Show
          cos Konstantin Boudnik added a comment - Well, in the last of 1.2.1 to be coming soon, let's revive this and try to make it happen in the current release, shall we? Roman Shaposhnik , do you want to share your interesting ideas? What Olaf Flebbe has proposed would most likely work, although the patch needs to be re-worked slightly to reflect the fact that we aren't using Vagrant anymore. However, I am strongly in favor of doing exactly this because otherwise we don't have an official way of testing our packages until the release repo is ready, which is a far too late.
          Hide
          rvs Roman Shaposhnik added a comment -

          Pushing it to 1.3.0 since I have a few ideas on how we can improve our CI even further there.

          Show
          rvs Roman Shaposhnik added a comment - Pushing it to 1.3.0 since I have a few ideas on how we can improve our CI even further there.
          Hide
          cos Konstantin Boudnik added a comment -

          Olaf Flebbe, Evans Ye: Guys, please consider committing this patch if it is ready. Thanks!

          Show
          cos Konstantin Boudnik added a comment - Olaf Flebbe , Evans Ye : Guys, please consider committing this patch if it is ready. Thanks!
          Hide
          evans_ye Evans Ye added a comment -

          If so, then we're going to commit this. I remember the url should be encoded first, for example , need to be encoded to %2C, otherwise somewhere on CentOS will fail.

          Show
          evans_ye Evans Ye added a comment - If so, then we're going to commit this. I remember the url should be encoded first, for example , need to be encoded to %2C , otherwise somewhere on CentOS will fail.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Thanks for elaborating. Indeed I prefer something like solution A, since otherwise we will almost never test the binary artifacts

          Show
          oflebbe Olaf Flebbe added a comment - Thanks for elaborating. Indeed I prefer something like solution A, since otherwise we will almost never test the binary artifacts
          Hide
          cos Konstantin Boudnik added a comment - - edited

          I do too prefer solution A, except that it will get applied to branch-1.1 and then merged back to the master , but it's an implementation detail. The reason A is better is simple: it doesn't limit our ability to re-point the master to the nightly/weekly builds later, if we need it.
          I will commit BIGTOP-2297 and merge to the master then. BIGTOP-2295 can be applied after if decided so. Thanks

          Show
          cos Konstantin Boudnik added a comment - - edited I do too prefer solution A, except that it will get applied to branch-1.1 and then merged back to the master , but it's an implementation detail. The reason A is better is simple: it doesn't limit our ability to re-point the master to the nightly/weekly builds later, if we need it. I will commit BIGTOP-2297 and merge to the master then. BIGTOP-2295 can be applied after if decided so. Thanks
          Hide
          evans_ye Evans Ye added a comment - - edited

          Make sense. Let me try to illustrate what will happened and sort of clarify the things:

          • Solution A: Let master be the bleeding edge builds.
            • Apply this patch to master and apply BIGTOP-2297 to branch-1.1, respectively. No follow up merge occurs.
          • Solution B: Let master use the latest release builds for cluster provisioning
            • Do not apply this patch. Apply BIGTOP-2297 to branch-1.1 and merge it back to master

          I prefer solution A. But here's an issue:
          The artifact download speed will be super slow when out of the US, which made the bleeding edge builds not so friendly to use for users. We can push daily builds to S3, but that's costly.
          Hence, I think just solution A will be OK, but we need to update all the document to tell users to switch to the latest release rather than using master.

          Show
          evans_ye Evans Ye added a comment - - edited Make sense. Let me try to illustrate what will happened and sort of clarify the things: Solution A: Let master be the bleeding edge builds. Apply this patch to master and apply BIGTOP-2297 to branch-1.1, respectively. No follow up merge occurs. Solution B: Let master use the latest release builds for cluster provisioning Do not apply this patch. Apply BIGTOP-2297 to branch-1.1 and merge it back to master I prefer solution A. But here's an issue: The artifact download speed will be super slow when out of the US, which made the bleeding edge builds not so friendly to use for users. We can push daily builds to S3, but that's costly. Hence, I think just solution A will be OK, but we need to update all the document to tell users to switch to the latest release rather than using master.
          Hide
          cos Konstantin Boudnik added a comment -

          It is even more interesting than that. We need to update these pointers for coming 1.1 release to target at (yet non-existent repos of 1.1 packages). On master, I think it is almost expected to have a latest stable build to be provisioned. If a user wants something more closely tested he would need to run provisioner from branch-1.1

          Makes sense?

          Show
          cos Konstantin Boudnik added a comment - It is even more interesting than that. We need to update these pointers for coming 1.1 release to target at (yet non-existent repos of 1.1 packages). On master, I think it is almost expected to have a latest stable build to be provisioned. If a user wants something more closely tested he would need to run provisioner from branch-1.1 Makes sense?
          Hide
          evans_ye Evans Ye added a comment -

          Hi Olaf Flebbe this is an interesting topic.
          Does user want to create a 1.1 SNAPSHOT cluster or user just want to use 1.1 Provisioner and Puppet features to create a cluster?
          This patch propose the former, however it could potentially fail due to the bleeding edge of the master build. I don't know whether its good or not for users who'd like to try it out...
          Another issue is that we need to apply a patch only to the release branch to change these defaults to stable, signed artifacts.
          It'd be easier if we can just fix the url to something like this:

          http://bigtop-repos.s3.amazonaws.com/releases/latest/debian/8/x86_64
          

          And use another customized vagrantconfig.yaml in our CI builds.

          Show
          evans_ye Evans Ye added a comment - Hi Olaf Flebbe this is an interesting topic. Does user want to create a 1.1 SNAPSHOT cluster or user just want to use 1.1 Provisioner and Puppet features to create a cluster? This patch propose the former, however it could potentially fail due to the bleeding edge of the master build. I don't know whether its good or not for users who'd like to try it out... Another issue is that we need to apply a patch only to the release branch to change these defaults to stable, signed artifacts. It'd be easier if we can just fix the url to something like this: http: //bigtop-repos.s3.amazonaws.com/releases/latest/debian/8/x86_64 And use another customized vagrantconfig.yaml in our CI builds.
          Hide
          oflebbe Olaf Flebbe added a comment -

          What do you think? Have to work out if yum works this way, too.

          Show
          oflebbe Olaf Flebbe added a comment - What do you think? Have to work out if yum works this way, too.

            People

            • Assignee:
              oflebbe Olaf Flebbe
              Reporter:
              oflebbe Olaf Flebbe
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development