Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
None
-
Normal
Description
We have been generating data (stress with CQL3 prepared) for CASSANDRA-4718 and found following problem almost in every SSTable generated (~200 GB of data and 821 SSTables).
We set up they keys to be 10 bytes in size (default) and population between 1 and 600000000.
Once I ran 'sstablekeys' on the generated SSTable files I got following exceptions:
There is a problem with sorting of normal looking keys:
30303039443538353645
30303039443745364242
java.io.IOException: Key out of order! DecoratedKey(-217680888487824985, 30303039443745364242) > DecoratedKey(-1767746583617597213, 30303039443437454333)
00000a30303033343933
37344400001388343933
java.io.IOException: Key out of order! DecoratedKey(5440473860101999581, 37344400001388343933) > DecoratedKey(-7565486415339257200, 30303033344639443137)
30303033354244363031
30303033354133423742
java.io.IOException: Key out of order! DecoratedKey(2687072396429900180, 30303033354133423742) > DecoratedKey(-7838239767410066684, 30303033354145344534)
30303034313442354137
30343136353633340000
java.io.IOException: Key out of order! DecoratedKey(1516003874415400462, 30343136353633340000) > DecoratedKey(-9106177395653818217, 30303034313333444238)
30303035373044373435
30303035373044334631
java.io.IOException: Key out of order! DecoratedKey(-3645715702154616540, 30303035373044334631) > DecoratedKey(-4296696226469000945, 30303035373132364138)
And completely different ones:
30303041333745373543
7cd045c59a90d7587d8d
java.io.IOException: Key out of order! DecoratedKey(-3595402345023230196, 7cd045c59a90d7587d8d) > DecoratedKey(-5146766422778260690, 30303041333943303232)
30303033323144444144
30303033323346343932
java.io.IOException: Key out of order! DecoratedKey(7071845511166615635, 30303033323346343932) > DecoratedKey(5233296131921119414, 53d83e00000012287e03)
30303034314531374431
3806734b256c27e41ec2
java.io.IOException: Key out of order! DecoratedKey(-7720474642702543193, 3806734b256c27e41ec2) > DecoratedKey(-8072288379146044663, 30303034314136413343)
And sometimes there is no problem at all:
30303033353144463637
0000002a31b3b31a1c2f
5d616dd38211ebb5d6ec
44423645000013880000
00001388138844463744
30303033353143394343
It's worth to mention that we have got 22 timeout exceptions but number of out-of-order keys is much larger than that.