[CASSANDRA-15729] Jenkins Test Results Report in plaintext for ASF ML

• Type: Task
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.2.17, 3.0.21, 3.11.7, 4.0-beta1, 4.0
• Component/s: Build, CI
• Labels: CI Jenkins
• Change Category: Code Clarity
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/94aee924e2d6b57ce3ecf90d39114246c4f7ec70

Test and Documentation Plan

Ran via Circle CI to validate XML

• Assignee: Michael Semb Wever
• Reporter: Michael Semb Wever

Authors

Michael Semb Wever

Reviewers

David Capwell, Eduard Tudenhoefner, Michael Semb Wever

• Created: 15/Apr/20 10:35
• Updated: 23/Jan/21 10:15
• Resolved: 06/May/20 11:15

Description

The Jenkins pipeline builds now aggregate all test reports.

For example:

• https://ci-cassandra.apache.org/job/Cassandra-trunk/68/testReport/
• https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-trunk/detail/Cassandra-trunk/68/tests

But Jenkins can only keep a limited amount of build history, so those links are not permanent, can't be used as references, and don't help for bisecting and blame on regressions (and flakey tests) over a longer period of time.

The builds@ ML can provide a permanent record of test results.

This was first brought up in these two threads:

• https://lists.apache.org/thread.html/re8122e4fdd8629e7fbca2abf27d72054b3bc0e3690ece8b8e66f618b%40%3Cdev.cassandra.apache.org%3E
• https://lists.apache.org/thread.html/ra5f6aeea89546825fe7ccc4a80898c62f8ed57decabf709d81d9c720%40%3Cdev.cassandra.apache.org%3E

An example plaintext report, to demonstrate feasibility, is available here: https://lists.apache.org/thread.html/r80d13f7af706bf8dfbf2387fab46004c1fbd3917b7bc339c49e69aa8%40%3Cbuilds.cassandra.apache.org%3E

Hurdles:

• the ASF mailing lists won't accept html, attachments, or any message body over 1MB.
• packages are used as a differentiator in the final aggregated report. The cqlsh and dtests currently don't specify it. It needs to be added as a "dot-separated" prefix to the testsuite and testcase name.

Issue Links

• is related to: CASSANDRA-15875 Jenkins pipeline small fixes: slack notification body, StabilityTestDataPublisher, archive cassandra-test-report.txt
• is related to: CASSANDRA-16401 Remove the Jenkins plaintext reports