[CASSANDRA-3536] Assertion error during bootstraping cassandra

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.0.6
• Labels: compaction

Environment

RHEL6 linux x86-64 guest on ESXi 5.0 (host), kernel: 2.6.32-71.24.1.el6.x86_64, 8 cpu, 20GB RAM

• Severity: Normal
• Assignee: Jonathan Ellis
• Reporter: Ramesh Natarajan

Authors

Jonathan Ellis

Reviewers

Benjamin Coverston

• Created: 28/Nov/11 17:44
• Updated: 16/Apr/19 09:32
• Resolved: 02/Dec/11 15:47

Description

I have a 3 node cassandra cluster. I have RF set to 3 and do reads
and writes using QUORUM.

Here is my initial ring configuration

[root@CAP4-CNode1 ~]# /root/cassandra/bin/nodetool -h localhost ring
Address DC Rack Status State Load
Owns Token

113427455640312821154458202477256070484
10.19.104.11 datacenter1 rack1 Up Normal 1.66 GB
33.33% 0
10.19.104.12 datacenter1 rack1 Up Normal 1.06 GB
33.33% 56713727820156410577229101238628035242
10.19.104.13 datacenter1 rack1 Up Normal 1.61 GB
33.33% 113427455640312821154458202477256070484

I want to add 10.19.104.14 to the cluster.

I edited the 10.19.104.14 cassandra.yaml file and set the token to
127605887595351923798765477786913079296 and set auto_bootstrap to
true.

When I started cassandra I am getting Assertion Error.

thanks
Ramesh

[root@CAP4-CNode4 cassandra]# INFO 10:29:46,093 Logging initialized
INFO 10:29:46,099 JVM vendor/version: Java HotSpot(TM) 64-Bit Server
VM/1.6.0_25
INFO 10:29:46,100 Heap size: 8304721920/8304721920
INFO 10:29:46,100 Classpath:
bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.0.2.jar:bin/../lib/apache-cassandra-clientutil-1.0.2.jar:bin/../lib/apache-cassandra-thrift-1.0.2.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.6.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mx4j-examples.jar:bin/../lib/mx4j-impl.jar:bin/../lib/mx4j.jar:bin/../lib/mx4j-jmx.jar:bin/../lib/mx4j-remote.jar:bin/../lib/mx4j-rimpl.jar:bin/../lib/mx4j-rjmx.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/jamm-0.2.5.jar
INFO 10:29:48,713 JNA mlockall successful
INFO 10:29:48,726 Loading settings from
file:/root/apache-cassandra-1.0.2/conf/cassandra.yaml
INFO 10:29:48,883 DiskAccessMode 'auto' determined to be mmap,
indexAccessMode is mmap
INFO 10:29:48,898 Global memtable threshold is enabled at 2640MB
INFO 10:29:49,203 Couldn't detect any schema definitions in local storage.
INFO 10:29:49,204 Found table data in data directories. Consider
using the CLI to define your schema.
INFO 10:29:49,220 Creating new commitlog segment
/var/lib/cassandra/commitlog/CommitLog-1321979389220.log
INFO 10:29:49,227 No commitlog files found; skipping replay
INFO 10:29:49,230 Cassandra version: 1.0.2
INFO 10:29:49,230 Thrift API version: 19.18.0
INFO 10:29:49,230 Loading persisted ring state
INFO 10:29:49,235 Starting up server gossip
INFO 10:29:49,259 Enqueuing flush of
Memtable-LocationInfo@122130810(192/240 serialized/live bytes, 4 ops)
INFO 10:29:49,260 Writing Memtable-LocationInfo@122130810(192/240
serialized/live bytes, 4 ops)
INFO 10:29:49,317 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db (300 bytes)
INFO 10:29:49,340 Starting Messaging Service on port 7000
INFO 10:29:49,349 JOINING: waiting for ring and schema information
INFO 10:29:50,759 Applying migration
4b0e20f0-1511-11e1-0000-c11bc95834d7 Add keyspace: MSA, rep
strategy:SimpleStrategy{}, durable_writes: true
INFO 10:29:50,761 Enqueuing flush of
Memtable-Migrations@1507565381(6744/8430 serialized/live bytes, 1 ops)
INFO 10:29:50,761 Writing Memtable-Migrations@1507565381(6744/8430
serialized/live bytes, 1 ops)
INFO 10:29:50,761 Enqueuing flush of
Memtable-Schema@1498835564(2889/3611 serialized/live bytes, 3 ops)
INFO 10:29:50,776 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-1-Data.db (6808 bytes)
INFO 10:29:50,777 Writing Memtable-Schema@1498835564(2889/3611
serialized/live bytes, 3 ops)
INFO 10:29:50,797 Completed flushing
/var/lib/cassandra/data/system/Schema-h-1-Data.db (3039 bytes)
INFO 10:29:50,814 Applying migration
4b6f2cb0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@1639d811[cfId=1000,ksName=MSA,cfName=modseq,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2f984f7d,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions=

{sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:50,815 Enqueuing flush of
Memtable-Migrations@948613108(7482/9352 serialized/live bytes, 1 ops)
INFO 10:29:50,816 Writing Memtable-Migrations@948613108(7482/9352
serialized/live bytes, 1 ops)
INFO 10:29:50,816 Enqueuing flush of
Memtable-Schema@421910828(3294/4117 serialized/live bytes, 3 ops)
INFO 10:29:50,831 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-2-Data.db (7546 bytes)
INFO 10:29:50,832 Writing Memtable-Schema@421910828(3294/4117
serialized/live bytes, 3 ops)
INFO 10:29:50,846 Completed flushing
/var/lib/cassandra/data/system/Schema-h-2-Data.db (3444 bytes)
INFO 10:29:50,854 Applying migration
4b8c9fc0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@1bd97d0d[cfId=1001,ksName=MSA,cfName=msgid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@63a0eec3,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
INFO 10:29:50,855 Enqueuing flush of
Memtable-Migrations@1520138062(7750/9687 serialized/live bytes, 1 ops)
INFO 10:29:50,856 Writing Memtable-Migrations@1520138062(7750/9687
serialized/live bytes, 1 ops)
INFO 10:29:50,856 Enqueuing flush of
Memtable-Schema@347459675(3630/4537 serialized/live bytes, 3 ops)
INFO 10:29:50,878 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-3-Data.db (7814 bytes)
INFO 10:29:50,879 Writing Memtable-Schema@347459675(3630/4537
serialized/live bytes, 3 ops)
INFO 10:29:50,894 Completed flushing
/var/lib/cassandra/data/system/Schema-h-3-Data.db (3780 bytes)
INFO 10:29:50,900 Applying migration
4ba1ae60-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@6a095b8a[cfId=1002,ksName=MSA,cfName=participants,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@c58f769,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
INFO 10:29:50,900 Enqueuing flush of
Memtable-Migrations@618337492(8194/10242 serialized/live bytes, 1 ops)
INFO 10:29:50,901 Writing Memtable-Migrations@618337492(8194/10242
serialized/live bytes, 1 ops)
INFO 10:29:50,902 Enqueuing flush of
Memtable-Schema@724860211(4020/5025 serialized/live bytes, 3 ops)
INFO 10:29:50,917 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-4-Data.db (8258 bytes)
INFO 10:29:50,918 Writing Memtable-Schema@724860211(4020/5025
serialized/live bytes, 3 ops)
INFO 10:29:50,925 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-3-Data.db')]
INFO 10:29:50,934 Completed flushing
/var/lib/cassandra/data/system/Schema-h-4-Data.db (4170 bytes)
INFO 10:29:50,935 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-3-Data.db')]
INFO 10:29:50,940 Applying migration
4bb4e840-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@318c69a9[cfId=1003,ksName=MSA,cfName=subinfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=5000.0,keyCacheSize=5000000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=14400,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@796cefa8,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10}

,compressionOptions={}]
INFO 10:29:50,941 Enqueuing flush of
Memtable-Migrations@1682081063(8618/10772 serialized/live bytes, 1
ops)
INFO 10:29:50,941 Writing Memtable-Migrations@1682081063(8618/10772
serialized/live bytes, 1 ops)
INFO 10:29:50,941 Enqueuing flush of
Memtable-Schema@1083461053(4427/5533 serialized/live bytes, 3 ops)
INFO 10:29:50,977 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-5-Data.db (8682 bytes)
INFO 10:29:50,978 Writing Memtable-Schema@1083461053(4427/5533
serialized/live bytes, 3 ops)
INFO 10:29:50,991 Compacted to
[/var/lib/cassandra/data/system/Schema-h-5-Data.db,]. 14,433 to
14,106 (~97% of original) bytes for 5 keys at 0.269051MB/s. Time:
50ms.
INFO 10:29:50,995 Completed flushing
/var/lib/cassandra/data/system/Schema-h-7-Data.db (4577 bytes)
INFO 10:29:51,000 Applying migration
4bc6e9a0-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@20b00ec2[cfId=1004,ksName=MSA,cfName=transactions,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@698f352,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions=

{sstable_size_in_mb=10},compressionOptions={}]
INFO 10:29:51,001 Enqueuing flush of
Memtable-Migrations@596545504(9027/11283 serialized/live bytes, 1 ops)
INFO 10:29:51,002 Writing Memtable-Migrations@596545504(9027/11283
serialized/live bytes, 1 ops)
INFO 10:29:51,003 Enqueuing flush of
Memtable-Schema@1686621532(4835/6043 serialized/live bytes, 3 ops)
INFO 10:29:51,029 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-7-Data.db (9091 bytes)
INFO 10:29:51,029 Writing Memtable-Schema@1686621532(4835/6043
serialized/live bytes, 3 ops)
INFO 10:29:51,031 Compacted to
[/var/lib/cassandra/data/system/Migrations-h-6-Data.db,]. 30,426 to
30,234 (~99% of original) bytes for 1 keys at 0.272013MB/s. Time:
106ms.
INFO 10:29:51,044 Completed flushing
/var/lib/cassandra/data/system/Schema-h-8-Data.db (4985 bytes)
INFO 10:29:51,049 Applying migration
4bd76460-1511-11e1-0000-c11bc95834d7 Add column family:
org.apache.cassandra.config.CFMetaData@4ab4faeb[cfId=1005,ksName=MSA,cfName=uid,cfType=Standard,comparator=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.BytesType),subcolumncomparator=<null>,comment=,rowCacheSize=0.0,keyCacheSize=1500000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=3600,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.SerializingCacheProvider@2fc5809e,mergeShardsChance=0.1,keyAlias=<null>,column_metadata={},compactionStrategyClass=class
org.apache.cassandra.db.compaction.LeveledCompactionStrategy,compactionStrategyOptions={sstable_size_in_mb=10}

,compressionOptions={}]
INFO 10:29:51,050 Enqueuing flush of
Memtable-Migrations@1333730706(9421/11776 serialized/live bytes, 1
ops)
INFO 10:29:51,050 Writing Memtable-Migrations@1333730706(9421/11776
serialized/live bytes, 1 ops)
INFO 10:29:51,051 Enqueuing flush of
Memtable-Schema@577668356(5236/6545 serialized/live bytes, 3 ops)
INFO 10:29:51,065 Completed flushing
/var/lib/cassandra/data/system/Migrations-h-9-Data.db (9485 bytes)
INFO 10:29:51,066 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-6-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-9-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-7-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Migrations-h-5-Data.db')]
INFO 10:29:51,066 Writing Memtable-Schema@577668356(5236/6545
serialized/live bytes, 3 ops)
INFO 10:29:51,081 Completed flushing
/var/lib/cassandra/data/system/Schema-h-9-Data.db (5386 bytes)
INFO 10:29:51,083 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-5-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-9-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-8-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/Schema-h-7-Data.db')]
INFO 10:29:51,114 Compacted to
[/var/lib/cassandra/data/system/Schema-h-10-Data.db,]. 29,054 to
28,727 (~98% of original) bytes for 8 keys at 0.913207MB/s. Time:
30ms.
INFO 10:29:51,144 Compacted to
[/var/lib/cassandra/data/system/Migrations-h-10-Data.db,]. 57,492 to
57,300 (~99% of original) bytes for 1 keys at 0.700584MB/s. Time:
78ms.
INFO 10:29:51,410 Node /10.19.104.13 is now part of the cluster
INFO 10:29:51,412 InetAddress /10.19.104.13 is now UP
INFO 10:29:51,414 Enqueuing flush of
Memtable-LocationInfo@709342045(35/43 serialized/live bytes, 1 ops)
INFO 10:29:51,415 Writing Memtable-LocationInfo@709342045(35/43
serialized/live bytes, 1 ops)
INFO 10:29:51,428 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db (89 bytes)
INFO 10:29:51,439 Node /10.19.104.12 is now part of the cluster
INFO 10:29:51,439 InetAddress /10.19.104.12 is now UP
INFO 10:29:51,441 Enqueuing flush of
Memtable-LocationInfo@1292444743(35/43 serialized/live bytes, 1 ops)
INFO 10:29:51,441 Writing Memtable-LocationInfo@1292444743(35/43
serialized/live bytes, 1 ops)
INFO 10:29:51,455 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db (89 bytes)
INFO 10:29:51,456 Node /10.19.104.11 is now part of the cluster
INFO 10:29:51,457 InetAddress /10.19.104.11 is now UP
INFO 10:29:51,459 Enqueuing flush of
Memtable-LocationInfo@1891328597(20/25 serialized/live bytes, 1 ops)
INFO 10:29:51,459 Writing Memtable-LocationInfo@1891328597(20/25
serialized/live bytes, 1 ops)
INFO 10:29:51,471 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db (74 bytes)
INFO 10:29:51,473 Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-4-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-1-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/LocationInfo-h-3-Data.db')]
INFO 10:29:51,497 Compacted to
[/var/lib/cassandra/data/system/LocationInfo-h-5-Data.db,]. 552 to
444 (~80% of original) bytes for 3 keys at 0.018410MB/s. Time: 23ms.
INFO 10:30:19,349 JOINING: getting bootstrap token
INFO 10:30:19,352 Enqueuing flush of
Memtable-LocationInfo@225265367(36/45 serialized/live bytes, 1 ops)
INFO 10:30:19,353 Writing Memtable-LocationInfo@225265367(36/45
serialized/live bytes, 1 ops)
INFO 10:30:19,364 Completed flushing
/var/lib/cassandra/data/system/LocationInfo-h-7-Data.db (87 bytes)
INFO 10:30:19,374 JOINING: sleeping 30000 ms for pending range setup
INFO 10:30:49,375 JOINING: Starting to bootstrap...
ERROR 10:31:13,444 Fatal exception in thread Thread[Thread-49,5,main]
java.lang.AssertionError
at org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:178)
at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:141)
at org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:466)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:275)
at org.apache.cassandra.db.DataTracker.addSSTables(DataTracker.java:237)
at org.apache.cassandra.db.DataTracker.addStreamedSSTable(DataTracker.java:242)
at org.apache.cassandra.db.ColumnFamilyStore.addSSTable(ColumnFamilyStore.java:922)
at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:141)
at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:102)
at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:184)
at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)