[CASSANDRA-12001] nodetool stopdaemon doesn't stop cassandra gracefully

• Type: Bug
• Status: Open
• Priority: Low
• Resolution: Unresolved
• Component/s: Tool/nodetool
• Labels: lhf

Environment

Ubuntu: Linux 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Cassandra Version :
cassandra -v
2.1.2

• Severity: Low
• Complexity: Low Hanging Fruit
• Assignee: Unassigned
• Reporter: Anshu Vajpayee
• Created: 13/Jun/16 19:25
• Updated: 24/Jul/19 23:04

Description

As per general opinion, nodetool stopdaemon should perform graceful shutdown rater than crash killing of cassandra daemon .
It doesn't flush the memtables and also it doesn't stop the thrift and CQL connection interfaces before crashing/stopping the node. It directly calls SIGTERM on process as simple as kill -15/ctrl + c.

1. created a table like as below:

cqlsh:test_ks> create table t2(id1 int, id2 text, primary key(id1));
cqlsh:test_ks>
cqlsh:test_ks> insert into t2(id1,id2) values (1,'a');
cqlsh:test_ks> insert into t2(id1,id2) values (2,'a');
cqlsh:test_ks> insert into t2(id1,id2) values (3,'a');
cqlsh:test_ks> select * from t2;

id1 | id2
----+----
1 | a
2 | a
3 | a

2.Flush the memtable manually using nodetool flush

student@cascor:~/node1/apache-cassandra-2.1.2/bin$ nodetool flush
student@cascor:~/node1/apache-cassandra-2.1.2/bin$ cd ../data/data/test_ks/t2-a671f6b0319a11e6a91ae3263299699d/
student@cascor:~/node1/apache-cassandra-2.1.2/data/data/test_ks/t2-a671f6b0319a11e6a91ae3263299699d$ ls -ltr
total 36
rw-rw-r- 1 student student 16 Jun 13 12:14 test_ks-t2-ka-1-Filter.db
rw-rw-r- 1 student student 54 Jun 13 12:14 test_ks-t2-ka-1-Index.db
rw-rw-r- 1 student student 93 Jun 13 12:14 test_ks-t2-ka-1-Data.db
rw-rw-r- 1 student student 91 Jun 13 12:14 test_ks-t2-ka-1-TOC.txt
rw-rw-r- 1 student student 80 Jun 13 12:14 test_ks-t2-ka-1-Summary.db
rw-rw-r- 1 student student 4442 Jun 13 12:14 test_ks-t2-ka-1-Statistics.db
rw-rw-r- 1 student student 10 Jun 13 12:14 test_ks-t2-ka-1-Digest.sha1
rw-rw-r- 1 student student 43 Jun 13 12:14 test_ks-t2-ka-1-CompressionInfo.db

3. Make few more changes on table t2

cqlsh:test_ks> insert into t2(id1,id2) values (5,'a');
cqlsh:test_ks> insert into t2(id1,id2) values (6,'a');
cqlsh:test_ks> insert into t2(id1,id2) values (7,'a');
cqlsh:test_ks> insert into t2(id1,id2) values (8,'a');
cqlsh:test_ks> select * from t2;

id1 | id2
----+----
5 | a
1 | a
8 | a
2 | a
7 | a
6 | a
3 | a

4. Stopping the node using nodetool stopdaemon
student@cascor:~$ nodetool stopdaemon
Cassandra has shutdown.
error: Connection refused
– StackTrace –
java.net.ConnectException: Connection refused

5. No new version of SStables . Reason stopdaemon doesn't run nodetool flush/drain before actually stopping daemon.

student@cascor:~/node1/apache-cassandra-2.1.2/data/data/test_ks/t2-a671f6b0319a11e6a91ae3263299699d$ ls -ltr
total 36
rw-rw-r- 1 student student 16 Jun 13 12:14 test_ks-t2-ka-1-Filter.db
rw-rw-r- 1 student student 54 Jun 13 12:14 test_ks-t2-ka-1-Index.db
rw-rw-r- 1 student student 93 Jun 13 12:14 test_ks-t2-ka-1-Data.db
rw-rw-r- 1 student student 91 Jun 13 12:14 test_ks-t2-ka-1-TOC.txt
rw-rw-r- 1 student student 80 Jun 13 12:14 test_ks-t2-ka-1-Summary.db
rw-rw-r- 1 student student 4442 Jun 13 12:14 test_ks-t2-ka-1-Statistics.db
rw-rw-r- 1 student student 10 Jun 13 12:14 test_ks-t2-ka-1-Digest.sha1
rw-rw-r- 1 student student 43 Jun 13 12:14 test_ks-t2-ka-1-CompressionInfo.db
student@cascor:~/node1/apache-cassandra-2.1.2/data/data/test_ks/t2-a671f6b0319a11e6a91ae3263299699d$