[CASSANDRA-16415] Digest mismatches during upgrade

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.25, 3.11.11, 4.0-rc1, 4.0
• Component/s: Consistency/Coordination, Test/dtest/java
• Bug Category: Correctness - Transient Incorrect Response
• Severity: Normal
• Complexity: Normal
• Discovered By: Unit Test
• Platform: All
• Since Version: 3.0.14

Source Control Link

https://github.com/apache/cassandra/commit/865b67b21d326728936f0b80681129c73a2e374a

Test and Documentation Plan

run regression + new tests

• Assignee: Jacek Lewandowski
• Reporter: Yifan Cai

Authors

Jacek Lewandowski

Reviewers

Andres de la Peña, Yifan Cai

• Created: 02/Feb/21 05:12
• Updated: 19/Nov/21 06:49
• Resolved: 01/Mar/21 12:57

Description

The test has been failing and can always be reproduced in the recent CI. 

Stack trace: 

junit.framework.AssertionFailedError: Found Digest Mismatch
 at org.apache.cassandra.distributed.upgrade.MixedModeReadTest.checkTraceForDigestMismatch(MixedModeReadTest.java:89)
 at org.apache.cassandra.distributed.upgrade.MixedModeReadTest.lambda$mixedModeReadColumnSubsetDigestCheck$0(MixedModeReadTest.java:63)
 at org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:171)
 at org.apache.cassandra.distributed.upgrade.MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck(MixedModeReadTest.java:76) 

The initial investigation shows that 

• The test only fails in the setup phase of mixedModeReadColumnSubsetDigestCheck. The cluster version is Versions.Major.v3X
• The test failure is likely a consequence of CASSANDRA-15962. After dropping the commit in branch cassandra-3.11 and rebuild the dtest jar, the upgrade test can pass. Meanwhile, dropping the other commits does not help. 

Issue Links

• causes: CASSANDRA-16483 ColumnFilter::toString doesn't return a valid CQL fragment
• is duplicated by: CASSANDRA-16432 Fix flaky test mixedModeReadColumnSubsetDigestCheck - org.apache.cassandra.distributed.upgrade.MixedModeReadTest
• links to: PR 3.0
• links to: PR 3.11
• links to: PR trunk