[CASSANDRA-9749] CommitLogReplayer continues startup after encountering errors

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.2.1, 3.0 beta 1
• Severity: Normal
• Assignee: Dmitry Konstantinov
• Reporter: Blake Eggleston

Authors

Dmitry Konstantinov

Reviewers

Ariel Weisberg

• Created: 07/Jul/15 17:20
• Updated: 21/Apr/20 15:02
• Resolved: 20/Aug/15 18:33

Description

There are a few places where the commit log recovery method either skips sections or just returns when it encounters errors.

Specifically if it can't read the header here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L298

Or if there are compressor problems here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L314 and here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L366

Whether these are user-fixable or not, I think we should require more direct user intervention (ie: fix what's wrong, or remove the bad file and restart) since we're basically losing data.

Issue Links

• is related to: CASSANDRA-7125 Fail to start by default if Commit Log fails to validate any messages
• relates to: CASSANDRA-6364 There should be different disk_failure_policies for data and commit volumes or commit volume failure should always cause node exit