Uploaded image for project: 'ActiveMQ Apollo (Retired)'
  1. ActiveMQ Apollo (Retired)
  2. APLO-359

Cannot shut down when stuck in startup

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Won't Do
    • 1.6
    • None
    • apollo-broker, apollo-cli
    • None
    • Ubuntu 14.04

    Description

      Steps to reproduce:

      1. In Terminal A, start ApolloMQ via apollo-cli.
      2. In Terminal B, do the same thing with the same options from the same working directory. Notice the log message: `WARN | DB operation failed. (entering recovery mode): java.io.IOException: File 'message-broker/data/lock' could not be locked.`
      3. In Terminal C, kill (SIGTERM) the ApolloMQ you created in Terminal B.

      Expected results: the process from Terminal B dies.

      Actual results: the process from Terminal B stays open, and only a SIGKILL can kill it. That's because the shutdown hook waits for the broker to stop, but the broker won't stop because it never started.

      Apollo's shutdown hook is a huge sysadmin headache for us. It hurts us in two places:

      A) On our cloud server, Upstart sends a SIGTERM and waits before sending a final SIGKILL. That means it takes an excessive amount of time for us to restart Apollo: we need to "stop; sleep 6; start" or we can't know the restart will succeed. Sure, we can solve this with extra sysadminning that sends SIGKILL judiciously; but if we're supposed to SIGKILL Apollo, then why is there a shutdown hook in the first place?

      B) We distribute a download-and-run parent program that invokes and monitors Apollo. This is a cross-platform Java program, so it can only send a SIGTERM. If Apollo doesn't shut down from a SIGTERM, it stays running forever, and it blocks new Apollo processes from starting.

      Attachments

        Activity

          People

            Unassigned Unassigned
            adamhooper Adam Hooper
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment