[CASSANDRA-17792] Fix race condition on updating cdc size and advancing to next segment

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0.6, 4.1-beta1, 5.0-alpha1, 5.0
• Component/s: CI
• Bug Category: Correctness
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 4.0

Source Control Link

https://github.com/apache/cassandra/commit/84e10d0cad5e34ef50d6951ca33714dfde0caa53

Test and Documentation Plan

ci

• Assignee: Yifan Cai
• Reporter: Ekaterina Dimitrova

Authors

Yifan Cai

Reviewers

Josh McKenzie

• Created: 02/Aug/22 15:41
• Updated: 12/Sep/23 13:03
• Resolved: 16/Aug/22 22:32

Description

org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest is a bit flaky on [trunk. 

As this run shows it was flaky since it was introduced a month ago as part of

CASSANDRA-17666 but the flakiness is so low that if we don't run it in a loop it is hard to hit it. 

Both tests in the test class can fail with the same exception:

org.apache.cassandra.distributed.shared.ShutdownException: Uncaught exceptions were thrown during test at org.apache.cassandra.distributed.impl.AbstractCluster.checkAndResetUncaughtExceptions(AbstractCluster.java:1056) at org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1042) at org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest.testCDCOnRepairEnabled(ToggleCDCOnRepairEnabledTest.java:95) at org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest.testCDCOnRepairIsEnabled(ToggleCDCOnRepairEnabledTest.java:40) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Suppressed: java.lang.NullPointerException at org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC$CDCSizeTracker.recalculateOverflowSize(CommitLogSegmentManagerCDC.java:390) at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81) at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47) at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829)

CC ycai , jmckenzie 

Issue Links

• Discovered while testing: CASSANDRA-17790 compaction_thoughput default value is different in cassandra.yaml and the Config class
• is caused by: CASSANDRA-17666 Option to disable write path during streaming for CDC enabled tables