[CASSANDRA-18133] In-tree build scripts

• Type: Task
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-alpha1, 5.0
• Component/s: CI
• Epic Link: repeatable_ci-cassandra
• Change Category: Quality Assurance
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra-builds/commit/de1e2375426429c0ec917352f478adfab3867234 https://github.com/apache/cassandra/commit/f5df4b219e063cb24b9cc0c22b6e614506b8d903

Test and Documentation Plan

lots of hallway testing.
custom jenkins jobs.

• Assignee: Michael Semb Wever Assign to me
• Reporter: Michael Semb Wever

Authors

Michael Semb Wever

Reviewers

Brandon Williams, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic

• Created: 04/Jan/23 12:07
• Updated: 19/Oct/23 08:18
• Resolved: 12/Jul/23 11:29

Description

Bring the artifact/deb/rpm build scripts (and associated docker images) from cassandra-builds repo to the .build directory.

The declarative Jenkinsfile can then directly declare the artifacts jobs in its pipeline. And the packaging jobs can be separated and run in parallel. EDIT: this bit has been moved to CASSANDRA-18594

This addresses the epic's stated existing problems:

• difficult to pre-commit test jenkins and cassandra-build changes,
• CI development efforts is split between ci-cassandra and circleci, despite ci-cassandra being our canonical and non-commercial CI,
• lacking parity of what is tested between ci-cassandra and circleci
• cassandra-builds as a separate repo (without release branches matching in-tree) adds complexity to changing matrix values (jdks, pythons, dist)
• mixture of jenkins dsl groovy, declarative and scripting pipeline.
• different pre-commit and post-commit jenkins pipelines are used.

In addition it addresses:

• stage jobs don't always running on the same SHA as the pipeline's run,
• infra issues around networking, specifically git cloning additional cassandra-builds repository,
• a more readable Jenkinsfile
• more UX friendly build and test scripts

Issue Links

• blocks: CASSANDRA-18594 Standalone Jenkinsfile
• is blocked by: CASSANDRA-17869 Add JDK17 option to cassandra-builds (build-scripts and jenkins dsl) and on jenkins agents
• is blocked by: CASSANDRA-18179 Update build.xml for experimental JDK17 use
• is blocked by: INFRA-24705 Create Cassandra dockerhub repositories
• is related to: CASSANDRA-16895 Build with Java 17
• is related to: CASSANDRA-17932 create a multiplexer job for Jenkins
• relates to: CASSANDRA-18565 Test pipeline does not always copy to nightlies
• relates to: CASSANDRA-18017 Jenkins: Consider using the no-build-test flag
• links to: Github PR