[CASSANDRA-6987] sstablesplit fails in 2.1

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1 beta2
• Labels: qa-resolved

Environment

Debian Testing/Jessie
Oracle JDK 1.7.0_51
c*-2.1 branch, commit 5ebadc11e36749e6479f9aba19406db3aacdaf41

• Severity: Normal
• Assignee: Benedict Elliott Smith
• Reporter: Michael Shuler

Authors

Benedict Elliott Smith

Reviewers

Jonathan Ellis

• Tester: Michael Shuler
• Created: 07/Apr/14 02:57
• Updated: 16/Apr/19 09:31
• Resolved: 25/Apr/14 23:05

Description

sstablesplit dtest began failing in 2.1 at http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/95/ triggered by http://cassci.datastax.com/job/cassandra-2.1/186/
repro:

(cassandra-2.1)mshuler@hana:~/git/cassandra$ ./bin/cassandra >/dev/null 
(cassandra-2.1)mshuler@hana:~/git/cassandra$ ./tools/bin/cassandra-stress write n=1000000
Created keyspaces. Sleeping 1s for propagation.
Warming up WRITE with 50000 iterations...
Connected to cluster: Test Cluster
Datatacenter: datacenter1; Host: localhost/127.0.0.1; Rack: rack1
Sleeping 2s...
Running WRITE with 50 threads  for 1000000 iterations
ops       ,    op/s,   key/s,    mean,     med,     .95,     .99,    .999,     max,   time,   stderr
26836     ,   26830,   26830,     2.0,     1.1,     4.0,    20.8,   131.4,   207.4,    1.0,  0.00000
64002     ,   36236,   36236,     1.4,     0.8,     4.2,    13.8,    41.3,   234.8,    2.0,  0.00000
105604    ,   38188,   38188,     1.3,     0.8,     3.2,    10.6,    78.4,    93.7,    3.1,  0.10546
156179    ,   36750,   36750,     1.4,     0.9,     2.9,     8.8,   117.0,   139.8,    4.5,  0.08482
202092    ,   40487,   40487,     1.2,     0.9,     2.9,     7.3,    45.6,   122.5,    5.6,  0.07231
246947    ,   40583,   40583,     1.2,     0.8,     3.0,     7.6,    98.2,   152.1,    6.7,  0.07056
290186    ,   39867,   39867,     1.3,     0.8,     2.6,     8.9,   113.3,   126.4,    7.8,  0.06391
331609    ,   40155,   40155,     1.2,     0.8,     3.1,     8.7,    99.1,   124.9,    8.8,  0.05731
371813    ,   38742,   38742,     1.3,     0.8,     3.1,     9.2,   117.2,   123.9,    9.9,  0.05153
416853    ,   40024,   40024,     1.2,     0.8,     3.2,     8.1,    70.4,   119.8,   11.0,  0.04634
458389    ,   39045,   39045,     1.3,     0.8,     3.2,     9.1,   106.4,   135.9,   12.1,  0.04236
511323    ,   36513,   36513,     1.4,     0.8,     3.3,     9.2,   120.2,   161.0,   13.5,  0.03883
549872    ,   34296,   34296,     1.5,     0.9,     3.4,    11.5,   106.7,   132.7,   14.6,  0.03678
589405    ,   34535,   34535,     1.4,     0.9,     2.9,    10.6,   106.2,   147.9,   15.8,  0.03607
633225    ,   39472,   39472,     1.3,     0.8,     3.0,     7.6,   106.3,   125.1,   16.9,  0.03374
672751    ,   38251,   38251,     1.3,     0.8,     3.0,     8.0,    94.7,   157.5,   17.9,  0.03193
714762    ,   38047,   38047,     1.3,     0.8,     3.0,     9.3,   102.6,   167.8,   19.0,  0.03001
756629    ,   38080,   38080,     1.3,     0.8,     3.2,     8.8,   101.7,   117.4,   20.1,  0.02847
802981    ,   38955,   38955,     1.3,     0.8,     3.0,     9.1,   105.2,   164.6,   21.3,  0.02708
847262    ,   38817,   38817,     1.3,     0.7,     3.2,     9.8,   112.1,   137.4,   22.5,  0.02581
887639    ,   38403,   38403,     1.3,     0.8,     2.9,    10.0,    99.1,   147.8,   23.5,  0.02470
929362    ,   35056,   35056,     1.4,     0.8,     3.3,    11.5,   111.8,   149.3,   24.7,  0.02360
980996    ,   38247,   38247,     1.3,     0.8,     3.5,     8.3,    78.8,   129.0,   26.1,  0.02338
1000000   ,   39379,   39379,     1.2,     0.9,     3.1,     9.0,    29.4,    83.8,   26.5,  0.02238


Results:
real op rate              : 37673
adjusted op rate stderr   : 0
key rate                  : 37673
latency mean              : 1.3
latency median            : 0.8
latency 95th percentile   : 3.2
latency 99th percentile   : 10.4
latency 99.9th percentile : 92.1
latency max               : 234.8
Total operation time      : 00:00:26
END
(cassandra-2.1)mshuler@hana:~/git/cassandra$ ./bin/nodetool compact Keyspace1
(cassandra-2.1)mshuler@hana:~/git/cassandra$ ./bin/sstablesplit /var/lib/cassandra/data/Keyspace1/Standard1-*/Keyspace1-Standard1-ka-2-Data.db
Exception in thread "main" java.lang.AssertionError
        at org.apache.cassandra.db.Keyspace.openWithoutSSTables(Keyspace.java:104)
        at org.apache.cassandra.tools.StandaloneSplitter.main(StandaloneSplitter.java:108)

There are no errors in system.log.

Issue Links

• depends upon: CASSANDRA-6916 Preemptive opening of compaction result