Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.1, 1.3.0
    • Component/s: build
    • Labels:
      None

      Description

      The CI Jobs are a little bit messy:

      • The run with user id 0
      • They depend on bind mounts to container
      • The generate files with user id 0 in the source directory
      • Have to be filled into jenkins command line

      Propose a build script

       
         bigtop-ci/build.sh --os trunk-debian-8
      

      Does not use bind mounts, do run with user id jenkins within container,
      create source images which can be reused be subsequent builds, do not generate user id 0 files.

      1. BIGTOP-2679.2.patch
        2 kB
        Evans Ye
      2. BIGTOP-2679.1.patch
        2 kB
        Olaf Flebbe

        Activity

        Hide
        evans_ye Evans Ye added a comment -

        Yes you're right. Close as fixed.

        Show
        evans_ye Evans Ye added a comment - Yes you're right. Close as fixed.
        Hide
        cos Konstantin Boudnik added a comment -

        Shall the ticket be closed if the code was pushed?

        Show
        cos Konstantin Boudnik added a comment - Shall the ticket be closed if the code was pushed?
        Hide
        evans_ye Evans Ye added a comment -

        Committed and pushed.
        Note that I've uploaded a new patch which has white spaces fixed.

        Show
        evans_ye Evans Ye added a comment - Committed and pushed. Note that I've uploaded a new patch which has white spaces fixed.
        Hide
        evans_ye Evans Ye added a comment -

        The patch is good during my tests.
        After discussed with Olaf this is by far the best solution for us. So I'm +1.

        Show
        evans_ye Evans Ye added a comment - The patch is good during my tests. After discussed with Olaf this is by far the best solution for us. So I'm +1.
        Hide
        evans_ye Evans Ye added a comment -

        OK. If this is also for user then I have a little concern about user experience since getting docker build involved here would be a bit confusing for users. Besides that, docker build takes a while to run.

        I'm wondering whether it's possible to
        1). start a container in detached mode
        2). cp the bigtop repo into container
        3). build
        4) cp results out

        Docker build cache does the great job when repo isn't changing. But on the developer/user side, the repo should change quite often. How do you think?

        Show
        evans_ye Evans Ye added a comment - OK. If this is also for user then I have a little concern about user experience since getting docker build involved here would be a bit confusing for users. Besides that, docker build takes a while to run. I'm wondering whether it's possible to 1). start a container in detached mode 2). cp the bigtop repo into container 3). build 4) cp results out Docker build cache does the great job when repo isn't changing. But on the developer/user side, the repo should change quite often. How do you think?
        Hide
        oflebbe Olaf Flebbe added a comment -

        I am not quite sure where your comment is heading to. Every user job can be used by CI. and vice-versa.

        Show
        oflebbe Olaf Flebbe added a comment - I am not quite sure where your comment is heading to. Every user job can be used by CI. and vice-versa.
        Hide
        oflebbe Olaf Flebbe added a comment -

        It was supposed to be open to users and the CI as well.

        Show
        oflebbe Olaf Flebbe added a comment - It was supposed to be open to users and the CI as well.
        Hide
        evans_ye Evans Ye added a comment -

        Olaf Flebbe I like the idea but I'd like to discuss the detail a bit more.
        It this supposed to be used by CI only or it's open to users as well?

        Show
        evans_ye Evans Ye added a comment - Olaf Flebbe I like the idea but I'd like to discuss the detail a bit more. It this supposed to be used by CI only or it's open to users as well?
        Hide
        rvs Roman Shaposhnik added a comment -

        Some of it has been done already. The rest can wait until 1.3.0 (unless anybody objects strongly)

        Show
        rvs Roman Shaposhnik added a comment - Some of it has been done already. The rest can wait until 1.3.0 (unless anybody objects strongly)
        Hide
        cos Konstantin Boudnik added a comment -

        If the only issue is the potential performance optimization, shall we just this in and improve in the consequent JIRA either in 1.2 or after it?

        Show
        cos Konstantin Boudnik added a comment - If the only issue is the potential performance optimization, shall we just this in and improve in the consequent JIRA either in 1.2 or after it?
        Hide
        evans_ye Evans Ye added a comment -

        I'm thinking we can have the image built, and then just run it everywhere.
        So the build time and runtime is separated. How do you think?

        Show
        evans_ye Evans Ye added a comment - I'm thinking we can have the image built, and then just run it everywhere. So the build time and runtime is separated. How do you think?
        Hide
        oflebbe Olaf Flebbe added a comment -

        Evans Ye It's designed do not have to care to have the image current. docker caches the whole build if already current.

        Show
        oflebbe Olaf Flebbe added a comment - Evans Ye It's designed do not have to care to have the image current. docker caches the whole build if already current.
        Hide
        evans_ye Evans Ye added a comment -

        Olaf Flebbe sorry for chiming in late.
        I'm curious why the design is to build the docker image first an then run it.
        Shall we separate the CI image builder and runner?

        Show
        evans_ye Evans Ye added a comment - Olaf Flebbe sorry for chiming in late. I'm curious why the design is to build the docker image first an then run it. Shall we separate the CI image builder and runner?
        Hide
        oflebbe Olaf Flebbe added a comment -

        Evans Ye: Yes this was the idea. Could you please review? If you decline, let's move it to 1.3

        Show
        oflebbe Olaf Flebbe added a comment - Evans Ye : Yes this was the idea. Could you please review? If you decline, let's move it to 1.3
        Hide
        cos Konstantin Boudnik added a comment -

        Shall we move this to 1.3?

        Show
        cos Konstantin Boudnik added a comment - Shall we move this to 1.3?
        Hide
        evans_ye Evans Ye added a comment -

        Is it possible for users to run build.sh to get things built?

        Show
        evans_ye Evans Ye added a comment - Is it possible for users to run build.sh to get things built?
        Hide
        oflebbe Olaf Flebbe added a comment -

        Continued work in BIGTOP-2679 branch

        Show
        oflebbe Olaf Flebbe added a comment - Continued work in BIGTOP-2679 branch
        Hide
        oflebbe Olaf Flebbe added a comment -

        My first proposal. Has to be improved a bit :

        • Add Apache header
        Show
        oflebbe Olaf Flebbe added a comment - My first proposal. Has to be improved a bit : Add Apache header

          People

          • Assignee:
            oflebbe Olaf Flebbe
            Reporter:
            oflebbe Olaf Flebbe
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development