[CASSANDRA-12509] Shutdown process triggered twice during if the node is drained

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.10, 3.10
• Component/s: Local/Startup and Shutdown
• Severity: Normal
• Assignee: Alex Petrov
• Reporter: Alex Petrov

Authors

Alex Petrov

Reviewers

Stefania Alborghetti

• Created: 19/Aug/16 18:58
• Updated: 16/Apr/19 09:30
• Resolved: 04/Oct/16 02:02

Description

If the node is drained, the StorageService#drain method is called, which triggers shutdown of mutation stage, messaging service, compaction, batchlog etc. In the end of this process, the node is moved to DRAINED status with the process still running.

When JVM is shutdown, the JVM shutdown hooks are ran, which are subscribed during the server initialisation: Runtime.getRuntime().addShutdownHook(drainOnShutdown); here.

I noticed this behaviour while reviewing CASSANDRA-12461, as if we'd like add custom pre and post-shutdown hooks, most likely it makes sense to run them once (or user might expect such behaviour).

Is this behaviour correct? Should we run whole shutdown process twice or just once in "drain" and no-op during JVM shutdown?

Issue Links

• is blocked by: CASSANDRA-11811 dtest failure in snapshot_test.TestArchiveCommitlog.test_archive_commitlog
• Is contained by: CASSANDRA-12461 Add hooks to StorageService shutdown