[CASSANDRA-5986] Cassandra hangs while reading saved cache

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Invalid
• Fix Version/s: 2.0.0
• Severity: Normal
• Assignee: Unassigned
• Reporter: cuser
• Created: 09/Sep/13 07:49
• Updated: 16/Apr/19 09:32
• Resolved: 13/Sep/13 22:31

Description

We've got a cluster ~6Tb size running on 5 nodes with vnodes(256) enabled.

After some cache heat-up one of the nodes was restarted and cassandra just hangs with no error. Last message:

cassandra.log:
...
INFO 11:12:27,649 Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
INFO 11:12:27,649 Opening /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
INFO 11:12:49,224 reading saved cache /cassandra/saved_caches/usertable-data-KeyCache-b.db

system.log:
...
INFO [SSTableBatchOpen:6] 2013-09-09 11:12:27,649 SSTableReader.java (line 213) Opening /cassandra/data/usertable/data/usertable-data-ja-1360 (6273771 bytes)
INFO [SSTableBatchOpen:7] 2013-09-09 11:12:27,649 SSTableReader.java (line 213) Opening /cassandra/data/usertable/data/usertable-data-ic-1339 (522628274110 bytes)
INFO [CompactionExecutor:1] 2013-09-09 11:12:49,224 AutoSavingCache.java (line 142) reading saved cache /cassandra/saved_caches/usertable-data-KeyCache-b.db

usertable-data-KeyCache-b.db size is about 3.5G.

All other nodes are restarting successfuly.