Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16276

Drain and/or shutdown might throw because of slow messaging service shutdown

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Bug Category:
      Code - Bug - Unclear Impact
    • Severity:
      Low
    • Complexity:
      Low Hanging Fruit
    • Discovered By:
      User Report
    • Platform:
      All
    • Impacts:
      None
    • Since Version:
    • Test and Documentation Plan:
      Hide

      None needed

      Show
      None needed

      Description

      If we invoke nodetool drain before shutdown, it sometimes fails to shut down messaging service in time (in this case - timing out the shutdown of the eventloopgroup by Netty). But, not before we manage to set isShutdown of StorageService to true, despite aborting further drain logic (including shutting down mutation stages).

      Then, via on shutdown hook, we invoke drain() method again, implicitly. We see that the mutation stage is not shutdown and proceed to assert that isShutdown == false, failing that assertion and triggering a second error log message.

      The patch merely ensures that any exception thrown by MS shutdown is captured so that drain logic can complete the first time around.

        Attachments

          Activity

            People

            • Assignee:
              aleksey Aleksey Yeschenko Assign to me
              Reporter:
              aleksey Aleksey Yeschenko
              Authors:
              Aleksey Yeschenko
              Reviewers:
              Aleksey Yeschenko, Alex Petrov, Marcus Eriksson

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment