Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: ci, general
    • Labels:
      None

      Description

      While this in not the blocker for 1.0 release, I think it's better to be done before we announce the 1.0 release because the FQDN of our CI master will be changed to the a new one.

        Issue Links

          Activity

          Hide
          evans_ye Evans Ye added a comment -

          Hi Roman Shaposhnik, Konstantin Boudnik mentioned that you have a backup of our CI configurations in http://bigtop01.cloudera.org:8080/, can you share it in the same way you send me docker credential? Appreciate that since I can't get in to that machine.

          Show
          evans_ye Evans Ye added a comment - Hi Roman Shaposhnik , Konstantin Boudnik mentioned that you have a backup of our CI configurations in http://bigtop01.cloudera.org:8080/ , can you share it in the same way you send me docker credential? Appreciate that since I can't get in to that machine.
          Hide
          evans_ye Evans Ye added a comment -

          I've successfully backup the xml and jobs//config.xml under /var/lib/jenkins from cloudera's donated CI master. The restore on new CI server is quite OK(jobs, configuration, etc) except the account settings. To be specifically, I can't login to my account using same username and password after on new CI server. I suspect the credential has been stored somewhere not named *.xml which I didn't backup. If someone knows please hint.
          I'll proceed first to bring jobs back online and push back the account issue. The worst case is to request everybody to register a new account and grant their access right.

          Show
          evans_ye Evans Ye added a comment - I've successfully backup the xml and jobs/ /config.xml under /var/lib/jenkins from cloudera's donated CI master. The restore on new CI server is quite OK(jobs, configuration, etc) except the account settings. To be specifically, I can't login to my account using same username and password after on new CI server. I suspect the credential has been stored somewhere not named *.xml which I didn't backup. If someone knows please hint. I'll proceed first to bring jobs back online and push back the account issue. The worst case is to request everybody to register a new account and grant their access right.
          Hide
          cos Konstantin Boudnik added a comment -

          Looks like the user info is stored under /var/lib/jenkins/users/ directory. I don't think we need to transfer all the accounts from the old server - perhaps only the ones that we know are active. I see there's a bunch of the accounts which hasn't been accessed for a year or more. I am pretty sure we can back them up, but perhaps not transfer them to the new server. Thoughts?

          Show
          cos Konstantin Boudnik added a comment - Looks like the user info is stored under /var/lib/jenkins/users/ directory. I don't think we need to transfer all the accounts from the old server - perhaps only the ones that we know are active. I see there's a bunch of the accounts which hasn't been accessed for a year or more. I am pretty sure we can back them up, but perhaps not transfer them to the new server. Thoughts?
          Hide
          evans_ye Evans Ye added a comment -

          You're right Konstantin Boudnik. I've created the new jenkins master and now everyone can use there account + password to login. I did some cleanup in new CI and have setup a matrix building job for all packages. Looks like some of the job is failing due to timeout and all the openSuSE job is failing due to incorrect JAVA_HOME setting. I'll bring up more CI jobs in next few days.

          Show
          evans_ye Evans Ye added a comment - You're right Konstantin Boudnik . I've created the new jenkins master and now everyone can use there account + password to login. I did some cleanup in new CI and have setup a matrix building job for all packages. Looks like some of the job is failing due to timeout and all the openSuSE job is failing due to incorrect JAVA_HOME setting. I'll bring up more CI jobs in next few days.
          Hide
          cos Konstantin Boudnik added a comment -

          Great! I will file an INFRA ticket to move the ci.bigtop.apache.org CNAME

          Show
          cos Konstantin Boudnik added a comment - Great! I will file an INFRA ticket to move the ci.bigtop.apache.org CNAME
          Hide
          evans_ye Evans Ye added a comment -

          The old CI master will be gone soon. Since the new CI is up and running now, I think we're fine.
          However, it looks like there're always some random builds failing.
          Most of them are failed due to the following download issue:

          * What went wrong:
          Error resolving plugin [id: 'de.undercouch.download', version: '2.0.0']
          > Could not GET 'https://plugins.gradle.org/api/gradle/2.7/plugin/use/de.undercouch.download/2.0.0'.
             > plugins.gradle.org
          

          Which I think is the server preventing DOS attack.
          To fix this, I'd like to have those plugin downloaded and stored in docker images directly, so that either or CI or users won't download things everytime when slave images created.
          Any other thoughts?

          Show
          evans_ye Evans Ye added a comment - The old CI master will be gone soon. Since the new CI is up and running now, I think we're fine. However, it looks like there're always some random builds failing. Most of them are failed due to the following download issue: * What went wrong: Error resolving plugin [id: 'de.undercouch.download', version: '2.0.0'] > Could not GET 'https: //plugins.gradle.org/api/gradle/2.7/plugin/use/de.undercouch.download/2.0.0'. > plugins.gradle.org Which I think is the server preventing DOS attack. To fix this, I'd like to have those plugin downloaded and stored in docker images directly, so that either or CI or users won't download things everytime when slave images created. Any other thoughts?
          Hide
          cos Konstantin Boudnik added a comment -

          Makes sense. At some point, when we have out own maven repo up-n-running, it will be cached there, I suppose.

          Show
          cos Konstantin Boudnik added a comment - Makes sense. At some point, when we have out own maven repo up-n-running, it will be cached there, I suppose.
          Hide
          cos Konstantin Boudnik added a comment -

          Shall we also start the CI master w/ HTTPS protocol enabled?
          Evans Ye, I will send you my ssh keys separately, could you please add them to the master host? In case you need some help, etc.

          Show
          cos Konstantin Boudnik added a comment - Shall we also start the CI master w/ HTTPS protocol enabled? Evans Ye , I will send you my ssh keys separately, could you please add them to the master host? In case you need some help, etc.
          Hide
          evans_ye Evans Ye added a comment -

          Based on my limited knowledge we need to generate certificate for ssl. And that certificate isn't available on users browser, or no?

          Show
          evans_ye Evans Ye added a comment - Based on my limited knowledge we need to generate certificate for ssl. And that certificate isn't available on users browser, or no?
          Hide
          evans_ye Evans Ye added a comment -

          Yup. Sounds valid.
          I think it's still better to include them in images so that users won't always download same things when creating a new slaves container.

          Show
          evans_ye Evans Ye added a comment - Yup. Sounds valid. I think it's still better to include them in images so that users won't always download same things when creating a new slaves container.
          Hide
          cos Konstantin Boudnik added a comment -

          I believe you're correct, perhaps INFRA can help us with that. I will check.

          Show
          cos Konstantin Boudnik added a comment - I believe you're correct, perhaps INFRA can help us with that. I will check.
          Hide
          evans_ye Evans Ye added a comment -

          This has been done.
          The new Jenkins master is at: http://ci.bigtop.apache.org:8080/

          Show
          evans_ye Evans Ye added a comment - This has been done. The new Jenkins master is at: http://ci.bigtop.apache.org:8080/

            People

            • Assignee:
              evans_ye Evans Ye
              Reporter:
              evans_ye Evans Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development