[CASSANDRA-6282] Scrub after changing compaction strategy breaks Cassandra

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Invalid
• Labels: compaction leveled

Environment

Tested on 1.1.12 and 1.2.11

• Severity: Normal
• Since Version: 1.1.12
• Assignee: Unassigned
• Reporter: André Cruz
• Created: 01/Nov/13 12:25
• Updated: 16/Apr/19 09:32
• Resolved: 01/Nov/13 13:00

Description

1 - Create table
2 - populate table
3 - Change compaction strategy from STCS to Leveled
update column family FileRevision with compaction_strategy = LeveledCompactionStrategy AND compaction_strategy_options =

{ sstable_size_in_mb:160 }

;
4 - Scrub
My files went from:
[andre@anarres ~/.ccm/LevelTest/node1/data/Disco/FileRevision]$ ls -lh
total 390816
rw-rr- 1 andre staff 24K Nov 1 12:16 Disco-FileRevision-ic-3-CompressionInfo.db
rw-rr- 1 andre staff 191M Nov 1 12:16 Disco-FileRevision-ic-3-Data.db
rw-rr- 1 andre staff 336B Nov 1 12:16 Disco-FileRevision-ic-3-Filter.db
rw-rr- 1 andre staff 900B Nov 1 12:16 Disco-FileRevision-ic-3-Index.db
rw-rr- 1 andre staff 4.3K Nov 1 12:16 Disco-FileRevision-ic-3-Statistics.db
rw-rr- 1 andre staff 153B Nov 1 12:16 Disco-FileRevision-ic-3-Summary.db
rw-rr- 1 andre staff 79B Nov 1 12:16 Disco-FileRevision-ic-3-TOC.txt
rw-rr- 1 andre staff 467B Nov 1 12:17 FileRevision.json

to:
[andre@anarres ~/.ccm/LevelTest/node1/data/Disco/FileRevision]$ ls -lh
total 390824
rw-rr- 1 andre staff 24K Nov 1 12:17 Disco-FileRevision-ic-4-CompressionInfo.db
rw-rr- 1 andre staff 191M Nov 1 12:17 Disco-FileRevision-ic-4-Data.db
rw-rr- 1 andre staff 176B Nov 1 12:17 Disco-FileRevision-ic-4-Filter.db
rw-rr- 1 andre staff 900B Nov 1 12:17 Disco-FileRevision-ic-4-Index.db
rw-rr- 1 andre staff 4.3K Nov 1 12:17 Disco-FileRevision-ic-4-Statistics.db
rw-rr- 1 andre staff 153B Nov 1 12:17 Disco-FileRevision-ic-4-Summary.db
rw-rr- 1 andre staff 79B Nov 1 12:17 Disco-FileRevision-ic-4-TOC.txt
rw-rr- 2 andre staff 467B Nov 1 12:17 FileRevision-old.json
rw-rr- 1 andre staff 467B Nov 1 12:17 FileRevision.json
drwxr-xr-x 3 andre staff 102B Nov 1 12:17 snapshots
5 - Try to list CF:
[default@Disco] list FileRevision limit 1;
Using default cell limit of 100
null
TimedOutException()
at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:12932)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:734)
at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:718)
at org.apache.cassandra.cli.CliClient.executeList(CliClient.java:1495)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:279)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:213)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:339)

Log:

ERROR [ReadStage:35] 2013-11-01 12:18:09,167 CassandraDaemon.java (line 191) Exception in thread Thread[ReadStage:35,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /Users/andre/.ccm/LevelTest/node1/data/Disco/FileRevision/Disco-FileRevision-ic-3-Data.db (No such file or directory)
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1618)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /Users/andre/.ccm/LevelTest/node1/data/Disco/FileRevision/Disco-FileRevision-ic-3-Data.db (No such file or directory)
at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:58)
at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1233)
at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:66)
at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1037)
at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:72)
at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1425)
at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1477)
at org.apache.cassandra.service.RangeSliceVerbHandler.executeLocally(RangeSliceVerbHandler.java:46)
at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1084)
at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1614)
... 3 more
Caused by: java.io.FileNotFoundException: /Users/andre/.ccm/LevelTest/node1/data/Disco/FileRevision/Disco-FileRevision-ic-3-Data.db (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:67)
at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:75)
at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:54)
... 12 more