Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Not Applicable
-
None
-
None
-
ghx-label-5
Description
The Impala public Jenkins service, jenkins.impala.io uses two different types of worker images:
1. Stock Ubuntu images published by the Ubuntu project.
These are pulled directly from Ubuntu's AWS account, they are booted with the default setting (including autoupdate every time the image boots), and all Impala-specific customizations are applied in startup and builds scripts.
The images are explicitly spcified using AWS AMI IDs; there is no "most recent" query or designator used.
2. AMIs prebaked by the Jenkins maintainers.
These images contain common preinstalled packages, like the Java runtime, Git and Docker; they usually have autoupdate turned off as well to decrease the latency between VM creation and the start of the build proper.
Both approaches can suffer from becoming stale:
1. Ubuntu publish new updates frequently, so more and more packages will be updated on each boot of an earlier stock image
2. The prebaked images will age, and by the very fact of skipping the update, become outdated. This is usually benign until it is not any more, at which time suddenly the world breaks because of some new change (like an external dependency bumping the required TLS version).
Both types should be kept up to date: for Ubuntu images a new AMI ID should be picked up from time to time; for type 2 the image should be rebaked using a base image. For both types the new image should be verified by passing a set of builds.
The process should be as automatic as possible.
Attachments
Issue Links
- relates to
-
IMPALA-8115 some jenkins workers slow to do real work due to dpkg lock conflicts
-
- Resolved
-
-
-
- Resolved
-