[CASSANDRA-6974] Replaying archived commitlogs isn't working

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1 rc1
• Labels: qa-resolved
• Severity: Normal
• Assignee: Benedict Elliott Smith
• Reporter: Ryan McGuire

Authors

Benedict Elliott Smith

Reviewers

Jonathan Ellis

• Tester: Michael Shuler
• Created: 02/Apr/14 16:02
• Updated: 16/Apr/19 09:31
• Resolved: 09/May/14 15:06

Description

I have a test for restoring archived commitlogs, which is not working in 2.1 HEAD. My commitlogs consist of 30,000 inserts, but system.log indicates there were only 2 mutations replayed:

INFO  [main] 2014-04-02 11:49:54,173 CommitLog.java:115 - Log replay complete, 2 replayed mutations

There are several warnings in the logs about bad headers and invalid CRCs:

WARN  [main] 2014-04-02 11:49:54,156 CommitLogReplayer.java:138 - Encountered bad header at position 0 of commit log /tmp/dtest
-mZIlPE/test/node1/commitlogs/CommitLog-4-1396453793570.log, with invalid CRC. The end of segment marker should be zero.

compare that to the same test run on 2.0, where it replayed many more mutations:

 INFO [main] 2014-04-02 11:49:04,673 CommitLog.java (line 132) Log replay complete, 35960 replayed mutations

I'll attach the system logs for reference.

Here is the dtest to reproduce this - (This currently relies on the fix for snapshots available in CASSANDRA-6965.)

Issue Links

• is blocked by: CASSANDRA-6978 Table creation does not add all the columns until schema is updated again
• is blocked by: CASSANDRA-7015 sstableloader NPE
• is related to: CASSANDRA-7846 Archive Commitlog Tests Failings