Accumulo
  1. Accumulo
  2. ACCUMULO-2985

MAC doesn't stop cleanly in 1.6.1-SNAPSHOT

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.2, 1.6.1, 1.7.0
    • Component/s: mini
    • Labels:
      None

      Description

      Using the following code to do some work-

      public class TestMACWithRealInstance {
        public static void main(String args[]) throws IOException, AccumuloException, AccumuloSecurityException, TableExistsException, InterruptedException {
          MiniAccumuloConfig macConfig = new MiniAccumuloConfig(new File("/tmp/mac"), "secret");
          macConfig.setNumTservers(2);
          MiniAccumuloCluster mac = new MiniAccumuloCluster(macConfig);
          mac.start();
          mac.getConnector("root", "secret").tableOperations().create("macCreated");
          mac.stop();
        }
      }
      

      It works fine against 1.6.0, but it seems broken against 01da4f4a8b14a125d3a2e29ef98dd044ab9ec75f after calling stop() it just sits in the terminal spewing messages about unable to connect to zookeeper

        Issue Links

          Activity

          Hide
          John Vines added a comment -

          Broken in 652253e01119e5c0445359c649316aa2d21dd718, which is the first commit in minicluster's code since 1.6.0

          Show
          John Vines added a comment - Broken in 652253e01119e5c0445359c649316aa2d21dd718, which is the first commit in minicluster's code since 1.6.0
          Hide
          Josh Elser added a comment -

          There's a single non-daemon thread. To implement ACCUMULO-2764, I wrapped the methods which stop the MAC sub processes in Callable's so we can get the timeout semantics. Sadly, this requires an Executor to get those timeout semantics. That Executor wasn't being stopped which introduced the bug that the above program outlines.

          I'm guessing that because Maven ultimately just ends the forked process, we never noticed that the surefire runner wasn't cleanly exiting on its own.

          Show
          Josh Elser added a comment - There's a single non-daemon thread. To implement ACCUMULO-2764 , I wrapped the methods which stop the MAC sub processes in Callable 's so we can get the timeout semantics. Sadly, this requires an Executor to get those timeout semantics. That Executor wasn't being stopped which introduced the bug that the above program outlines. I'm guessing that because Maven ultimately just ends the forked process, we never noticed that the surefire runner wasn't cleanly exiting on its own.
          Hide
          Josh Elser added a comment -

          Ensure that the ES is shutdown before stop() returns from MiniAccumuloCluster. Unit test ensures that the ES is in fact shutdown. If there is somehow pending actions (which should be impossible, afaict), they get logged at WARN.

          Show
          Josh Elser added a comment - Ensure that the ES is shutdown before stop() returns from MiniAccumuloCluster. Unit test ensures that the ES is in fact shutdown. If there is somehow pending actions (which should be impossible, afaict), they get logged at WARN.
          Hide
          Josh Elser added a comment -

          I'm reopening this because, after committing these changes, my jenkins instance has been repeatedly dying from (I think) running out of memory. I'm seeing loads of Accumuo and ZooKeeper processes hanging around. I'm guessing that perhaps the shutdown logic on the ES isn't right.

          Show
          Josh Elser added a comment - I'm reopening this because, after committing these changes, my jenkins instance has been repeatedly dying from (I think) running out of memory. I'm seeing loads of Accumuo and ZooKeeper processes hanging around. I'm guessing that perhaps the shutdown logic on the ES isn't right.
          Hide
          Josh Elser added a comment -

          It looked like this was fine up until it ran the VolumeIT which I had at least 4 accumulo clusters still running. I think the issue is that if the MAC is restarted after being stopped, the ES would still be stopped.

          Show
          Josh Elser added a comment - It looked like this was fine up until it ran the VolumeIT which I had at least 4 accumulo clusters still running. I think the issue is that if the MAC is restarted after being stopped, the ES would still be stopped.
          Hide
          Josh Elser added a comment -

          Make sure that the ES is not shutdown when a cluster is started, so that it can be used to stop the processes again.

          Show
          Josh Elser added a comment - Make sure that the ES is not shutdown when a cluster is started, so that it can be used to stop the processes again.

            People

            • Assignee:
              Josh Elser
              Reporter:
              John Vines
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h

                  Development