This was generated from latest "future bigtop" thread on mailing list, capturing core details here from Olaf Flebbe
cc: Evans Ye
However we have a substantial problem hidden deep in th CI „2.0“ approach using containers
You may know that we place artifacts (i.e. jars) we built with bigtop into the local maven cache ~/.m2. (look for mvn install in do-component-build). The idea is that later maven builds will pick these artifacts and use them rather downloading them from maven central.
Placing artifacts into ~/.m2 will not have any effect if we use CI containers the way we do now: The maven cache ~/.m2 is lost when the container ends.
[This triggered misfeature in JIRA https://issues.apache.org/jira/browse/BIGTOP-1893 , BTW: gradle rpm/apt behaved differently from a container build with artifacts from maven central.]
Option 1) Remove mvn install from all do-component-builds
+ We compile projects the way the upstream-developer does.
- local fixes and configurations will not propagated
If we do not try to reuse our build-artifacts within compile we have to ask ourself "why do we compile projects at all?“.
We can build a great test wether someone else has touched / manipulated the maven central cache if we compare artifacts, but is this the really the point of compiling ourselves?
Option 2) Use mvn install and reuse artifacts even in containers.
- Containers are not stateless any more
- We have to add depencies to CI jobs so they run in order
- single components may break the whole compile process.
- Compile does not scale any more
The way we do now "mvn install“ , simply tainting the maven cache seems not a really controlled way to propagate artifacts to me.
Option 3) Use 1) but reuse artifacts in packages by placing symlinks and dependencies between them.
- Packages will break with subtile problems if we do symlink artifacts from different releases.
Neither Option 1, Option 2 nor Option 3 seems a clever way to fix the problem. Would like to hear comments regarding this issue:
In my humble opinion we should follow Option 2 with all the grave consequences. But maybe reworking mvn install by placing the artifacts with a bigtop specific name / groupid into the maven cache and upload them to maven central .