Uploaded image for project: 'Brooklyn'
  1. Brooklyn
  2. BROOKLYN-264

Stop app while VM still being provisioned: vm is left running when app is expunged

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.0
    • 0.10.0
    • None

    Description

      A customer deployed an app to AWS, but while the VM was still starting up they stopped (and thus expunged) the app. The app disappeared from the Brooklyn web-console, but the starting VM was left behind in AWS.

      This is simple to reproduce:
      1. deploy a simple blueprint, such as:

      location: aws-ec2:us-east-1
      services:
      - type: org.apache.brooklyn.entity.machine.MachineEntity
      

      2. wait for the VM to appear in the AWS web-console (with state "initialising")
      3. call the stop effector on the top-level app.


      Looking at the start task that was executing at the time when stop was called, below is the thread's stack trace:

      Provisioning machine in JcloudsLocation[AWS Virginia:AAAAAAAAAAAAAAAAAAAA/aws-ec2:us-east-1@eyNrLIo5]
      
      Task[provisioning (AWS Virginia)]@MJITkjw0
      Submitted by SoftlyPresent[value=Task[start]@tKw0qJET]
      
      In progress, thread waiting (notify) on java.util.concurrent.CountDownLatch$Sync@2ed5be36
      At: org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:149)
          org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:214)
          org.jclouds.ec2.compute.EC2ComputeService.createNodesInGroup(EC2ComputeService.java:149)
          org.apache.brooklyn.location.jclouds.JcloudsLocation.obtainOnce(JcloudsLocation.java:726)
          org.apache.brooklyn.location.jclouds.JcloudsLocation.obtain(JcloudsLocation.java:616)
          org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:406)
          org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ObtainLocationTask.call(MachineLifecycleEffectorTasks.java:396)
          org.apache.brooklyn.util.core.task.Tasks.withBlockingDetails(Tasks.java:98)
          org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:380)
          org.apache.brooklyn.entity.software.base.lifecycle.MachineLifecycleEffectorTasks$ProvisionMachineTask.call(MachineLifecycleEffectorTasks.java:364)
          org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359)
          org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519)
      

      From this, we can see that we are still calling jclouds. This means that jclouds has not yet returned to Brooklyn the VM's id. It also means that the MachineEntity will not have been given a JcloudsSshMachineLocation instance.

      When stop is called on the MachineEntity, it doesn't have a machine location instance so it doesn't have anything to ask to stop. This is why the VM is left running.

      Attachments

        Activity

          People

            Unassigned Unassigned
            aled.sage Aled Sage
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: