Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2081

implement a nexus docker container for CI

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: build
    • Labels:
      None

      Description

      In order to cache all the maven dependencies we need a maven artifact proxy server.

      Lets create a docker OSS nexus server with some proxy repositories already preconfigured.

      Persistent data is handled by a data volume container see
      https://docs.docker.com/userguide/dockervolumes/

      The client config will be handled by something like, if client ip port 8081 is mapped to server ip port 8081

      cat >$HOME/.m2/settings.xml <<EOF
      <settings>
        <mirrors>
          <mirror>
            <id>conjars</id>
            <name>nexus conjars</name>
            <url>http://localhost:8081/nexus/content/repositories/conjars/</url>
            <mirrorOf>conjars</mirrorOf>
          </mirror>
          <mirror>
            <id>central</id>
            <name>nexus central</name>
            <url>http://localhost:8081/nexus/content/repositories/central/</url>
            <mirrorOf>central</mirrorOf>
          </mirror>
           <mirror>
            <id>repository.jboss.org</id>
            <name>nexus jboss</name>
          <url>http://localhost:8081/nexus/content/repositories/repository.jboss.org/</url>
            <mirrorOf>repository.jboss.org</mirrorOf>
          </mirror>
          <mirror>
            <id>apache snapshot</id>
            <name>apache snapshot</name>
            <url>http://localhost:8081/nexus/content/repositories/apache_snapshots/</url>
            <mirrorOf>apache snapshot</mirrorOf>
          </mirror>
          <mirror>
            <id>apache https snapshot</id>
            <url>http://localhost:8081/nexus/content/repositories/apache_snapshots.https/</url>
            <mirrorOf>apache.snapshots.https</mirrorOf>
          </mirror>
      	<mirror>
             <id>maven2-repository.atlassian</id>
             <url>http://localhost:8081/nexus/content/repositories/maven2-repository.atlassian/</url>
             <mirrorOf>maven2-repository.atlassian</mirrorOf>
          </mirror>
        </mirrors>
      </settings>
      EOF
      
      1. BIGTOP-2081.1.patch
        24 kB
        Olaf Flebbe
      2. BIGTOP-2081.2.patch
        3 kB
        Olaf Flebbe
      3. BIGTOP-2081.3.patch
        3 kB
        Olaf Flebbe
      4. BIGTOP-2081.4.patch
        3 kB
        Olaf Flebbe

        Activity

        Hide
        oflebbe Olaf Flebbe added a comment -

        A first WIP

        Show
        oflebbe Olaf Flebbe added a comment - A first WIP
        Hide
        oflebbe Olaf Flebbe added a comment -

        Found the "official" nexus container ... https://github.com/sonatype/docker-nexus

        Will use this one instead

        Show
        oflebbe Olaf Flebbe added a comment - Found the "official" nexus container ... https://github.com/sonatype/docker-nexus Will use this one instead
        Hide
        oflebbe Olaf Flebbe added a comment -

        Now a working patch with gradle support.

        Usage:

        Rewrite the CI job to

        if docker logs nexus >/dev/null ; then 
           :
        else
           docker run -d --name=nexus sonatype/nexus 
        fi
        
        docker run --rm -v `pwd`:/ws --workdir /ws --net=container:nexus  -e COMPONENTS=$COMPONENTS bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \
        bash -l -c 'A=rpm ; type -p dpkg && A=deb ; ./gradlew allclean ; ./gradlew -Dbuildnodeps=true configure-nexus $COMPONENTS-$A'
        

        Explanation:

        Test if a nexus container is already running.
        If not start nexus
        Now use a container internal network to connect build slaves to nexus.
        the gradle configure-nexus sets up several proxy repositories via the nexus REST API and writes a suitable settings.xml to pick up the repositories.
        You will find that a maven compilation will download most of the artifacts from http://localhost:8081

        The setup is secure, since the repository is not visible from the outside.

        Show
        oflebbe Olaf Flebbe added a comment - Now a working patch with gradle support. Usage: Rewrite the CI job to if docker logs nexus >/dev/ null ; then : else docker run -d --name=nexus sonatype/nexus fi docker run --rm -v `pwd`:/ws --workdir /ws --net=container:nexus -e COMPONENTS=$COMPONENTS bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \ bash -l -c 'A=rpm ; type -p dpkg && A=deb ; ./gradlew allclean ; ./gradlew -Dbuildnodeps= true configure-nexus $COMPONENTS-$A' Explanation: Test if a nexus container is already running. If not start nexus Now use a container internal network to connect build slaves to nexus. the gradle configure-nexus sets up several proxy repositories via the nexus REST API and writes a suitable settings.xml to pick up the repositories. You will find that a maven compilation will download most of the artifacts from http://localhost:8081 The setup is secure, since the repository is not visible from the outside.
        Hide
        evans_ye Evans Ye added a comment -

        I've poor knowledge on nexus. But the gradle code looks good.
        Just one more thing, there're two unrelated newline introduce in the patch.

        Show
        evans_ye Evans Ye added a comment - I've poor knowledge on nexus. But the gradle code looks good. Just one more thing, there're two unrelated newline introduce in the patch.
        Hide
        oflebbe Olaf Flebbe added a comment -

        rebased patch to current git

        Show
        oflebbe Olaf Flebbe added a comment - rebased patch to current git
        Hide
        oflebbe Olaf Flebbe added a comment -

        committed. Will change the CI a bit later

        Show
        oflebbe Olaf Flebbe added a comment - committed. Will change the CI a bit later
        Hide
        cos Konstantin Boudnik added a comment -

        This patch has a side effect of overwriting user's ~/.m2/settings.xml because of the following:

        repos.each { r->
          generate_nexus_tasks( r.name, r.url, r.snapshot)
        }
        

        This should be executed conditionally and turned off by default.

        Show
        cos Konstantin Boudnik added a comment - This patch has a side effect of overwriting user's ~/.m2/settings.xml because of the following: repos.each { r-> generate_nexus_tasks( r.name, r.url, r.snapshot) } This should be executed conditionally and turned off by default.
        Hide
        cos Konstantin Boudnik added a comment -

        Also, let's try not to have 150+ chars long lines. It is hard to read.

        Show
        cos Konstantin Boudnik added a comment - Also, let's try not to have 150+ chars long lines. It is hard to read.
        Hide
        oflebbe Olaf Flebbe added a comment -

        Hm, somehow gradle seems to work different than I expected (and tested). Will try to fix it soon.

        Show
        oflebbe Olaf Flebbe added a comment - Hm, somehow gradle seems to work different than I expected (and tested). Will try to fix it soon.
        Hide
        oflebbe Olaf Flebbe added a comment -

        Added another maven repo (for tez build), reformatted code and fixed the issue observed. (Mixed defining a closure and using it)

        Show
        oflebbe Olaf Flebbe added a comment - Added another maven repo (for tez build), reformatted code and fixed the issue observed. (Mixed defining a closure and using it)
        Hide
        oflebbe Olaf Flebbe added a comment -

        This one works

        Show
        oflebbe Olaf Flebbe added a comment - This one works
        Hide
        cos Konstantin Boudnik added a comment -

        Yeah, this one doesn't have side-effects indeed, Thank you!
        Let's get it in so we can fix CI's annoying network issues. Ideally, I would like to make this code a bit more prettier with separate definitions of the ports and stuff, instead of having everything hardcoded in the URL, but it is a very low priority at this point.

        Please commit!

        Show
        cos Konstantin Boudnik added a comment - Yeah, this one doesn't have side-effects indeed, Thank you! Let's get it in so we can fix CI's annoying network issues. Ideally, I would like to make this code a bit more prettier with separate definitions of the ports and stuff, instead of having everything hardcoded in the URL, but it is a very low priority at this point. Please commit!
        Hide
        oflebbe Olaf Flebbe added a comment -

        Committed, and changed CI job to:

        if docker logs nexus >/dev/null ; then 
           :
        else
           docker run -d --name=nexus sonatype/nexus 
        fi
        
        docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS  --net=container:nexus  bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \
        bash -l -c './gradlew allclean ; ./gradlew  configure-nexus $COMPONENTS-pkg'
        

        ... waiting ...

        Show
        oflebbe Olaf Flebbe added a comment - Committed, and changed CI job to: if docker logs nexus >/dev/ null ; then : else docker run -d --name=nexus sonatype/nexus fi docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS --net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \ bash -l -c './gradlew allclean ; ./gradlew configure-nexus $COMPONENTS-pkg' ... waiting ...
        Hide
        oflebbe Olaf Flebbe added a comment -

        BTW: I made a backup of the job Bigtop-trunk-packages-backup in case of emergency...

        Show
        oflebbe Olaf Flebbe added a comment - BTW: I made a backup of the job Bigtop-trunk-packages-backup in case of emergency...

          People

          • Assignee:
            oflebbe Olaf Flebbe
            Reporter:
            oflebbe Olaf Flebbe
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development