[CASSANDRA-9686] FSReadError and LEAK DETECTED after upgrading

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.9, 2.2.0, 3.0 alpha 1
• Component/s: Legacy/Local Write-Read Paths

Environment

Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55

• Severity: Normal
• Since Version: 2.2.0 rc1
• Assignee: Stefania Alborghetti
• Reporter: Andreas Schnitzerling

Authors

Stefania Alborghetti

Reviewers

Marcus Eriksson

• Created: 30/Jun/15 10:48
• Updated: 16/Apr/19 09:31
• Resolved: 13/Jul/15 17:32

Description

After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and LEAK DETECTED on start. Deleting the listed files, the failure goes away.

system.log

ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554 DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with 0 chunks encountered: java.io.DataInputStream@1c42271
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:117) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered: java.io.DataInputStream@1c42271
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174) ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
	... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866 was not released before the reference was garbage collected

Issue Links

• links to: 2.1 backport patch
• links to: 2.2 patch