[CASSANDRA-19833] Improve CQLTester to make it trivial to run the tests with different configs, and to add randomness to the test

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.1
• Component/s: Test/fuzz
• Change Category: Quality Assurance
• Complexity: Low Hanging Fruit
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/2c52a8be6fdac7305e433b7f28bd70596ef1444a

Test and Documentation Plan

improved testing

• Assignee: David Capwell
• Reporter: David Capwell

Authors

David Capwell

Reviewers

Caleb Rackliffe

• Created: 13/Aug/24 21:32
• Updated: 21/Aug/24 02:47
• Resolved: 21/Aug/24 02:47

Description

When creating a test many authors default to hard coding values as the barrier to make the test generic feels too high; to lower this barrier we should add an extension class to CQLTester that enables randomized testing without having to worry about which framework to use.

This should setup the single node cluster with different configs to improve coverage.

Here is a sample error message when a test fails

java.lang.AssertionError: Property error detected:
Config Seed: -3611771839852432544 -- To rerun do -Dcassandra.test.cqltester.fuzzed.seed.org.apache.cassandra.cql3.validation.operations.InsertTest=-3611771839852432544
Config:
	---
	partitioner: OrderPreservingPartitioner
	commitlog_sync: batch
	commitlog_disk_access_mode: mmap_index_only
	memtable_allocation_type: offheap_buffers
	sstable:
	  selected_format: bti
	disk_access_mode: mmap
	
Error:
	commitlog_disk_access_mode = mmap_index_only is not supported. Please use 'auto' when unsure.

And if the test fails

java.lang.AssertionError: Property error detected:
Config Seed: -5023694648830703272 -- To rerun do -Dcassandra.test.cqltester.fuzzed.seed.org.apache.cassandra.cql3.validation.operations.InsertTest=-5023694648830703272
Seed: -1088015559108807046 -- To rerun do -Dcassandra.test.cqltester.fuzzed.seed.org.apache.cassandra.cql3.validation.operations.InsertTest.testInsertZeroDuration=-1088015559108807046
Config:
	---
	partitioner: OrderPreservingPartitioner
	commitlog_sync: periodic
	commitlog_sync_period: 10000ms
	commitlog_disk_access_mode: direct
	memtable_allocation_type: offheap_objects
	sstable:
	  selected_format: bti
	disk_access_mode: mmap_index_only
Caused by: java.lang.NullPointerException

Issue Links

• links to: GH: trunk