[CASSANDRA-9851] Write Durability Failures Even During Batch Commit Mode

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate

Environment

Debian, x86_64, Kernel 3.16.7

• Severity: Normal
• Assignee: Unassigned Assign to me
• Reporter: Joel Knighton
• Created: 20/Jul/15 20:20
• Updated: 16/Apr/19 09:31
• Resolved: 21/Jul/15 16:11

Description

Reproducible as of a66863861136a29dc04d7bc3b319f9f8fae0f49f on trunk, as well as in other recent commits.

Durability of writes seems to be violated, even under batch commitlog mode. This issue was discovered by a test that adds a range of values to a CQL Set, with no deletes issued. The test is available here https://github.com/riptano/jepsen/blob/cassandra/cassandra/src/cassandra/collections/set.clj#L56.

During this write pattern, random nodes in the 5 node cluster are kill -9ed. Once all nodes have been brought back up, another read at CL.ALL is issued. This read fails to return values that have previously been successfully read from the cluster. This problem is not reproducible on 2.1.* or 2.2.

Log files from each node are attached.