Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-9626

Instance fails to start after unsuccesful compute offering upgrade.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 4.8.0
    • None
    • Management Server
    • Security Level: Public (Anyone can view this level - this is the default.)
    • None

    Description

      ISSUE
      ============
      Instance fails to start after unsuccesful compute offering upgrade.

      TROUBLESHOOTING
      ==================
      We observed VM instance get compute values "cpuNumber","cpuSpeed","memory" removed from table "user_vm_details", which cause instance fail to startup next time on XenServer

      mysql> select * from user_vm_details where vm_id=10;
      --------------------------------------------------------------------------------------------------

      id vm_id name value display

      --------------------------------------------------------------------------------------------------

      218 10 platform viridian:true;acpi:1;apic:true;pae:true;nx:true 1
      219 10 hypervisortoolsversion xenserver56 1
      220 10 Message.ReservedCapacityFreed.Flag true 1

      --------------------------------------------------------------------------------------------------
      3 rows in set (0.00 sec)

      Unexpected exception while executing org.apache.cloudstack.api.command.user.vm.ScaleVMCmd
      java.lang.NullPointerException
      at com.cloud.vm.UserVmManagerImpl.upgradeStoppedVirtualMachine(UserVmManagerImpl.java:953)
      at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1331)
      at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1271)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:616)
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
      at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
      at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
      at $Proxy169.upgradeVirtualMachine(Unknown Source)
      at org.apache.cloudstack.api.command.user.vm.ScaleVMCmd.execute(ScaleVMCmd.java:127)
      at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:167)
      at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:97)
      at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:543)
      at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
      at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
      at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
      at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:500)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:679)

      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) Select by Id SQL: SELECT user_vm_details.id, user_vm_details.vm_id, user_vm_details.name, user_vm_details.value, user_vm_details.display FROM user_vm_details WHERE user_vm_details.id = ?
      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) Table References: user_vm_details
      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) Insert SQLs:
      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) INSERT INTO user_vm_details (user_vm_details.vm_id, user_vm_details.name, user_vm_details.value, user_vm_details.display) VALUES (?, ?, ?, ?)
      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) Delete SQLs
      2015-02-09 15:23:46,578 TRACE [c.c.u.d.GenericDaoBase] (main:null) DELETE FROM user_vm_details WHERE user_vm_details.id= ?

      REPRO STEPS
      ==================
      1. Set global setting enable.dynamic.scale.vm to true
      2. Create a custom Compute Offerings A
      3. Create a VM instance apply A, ie. cpuNumber=1,cpuSpeed=1000,memory=512M
      4. Create another custom Compute Offerings B
      5. Change service offering to B, ie. cpuNumber=2,cpuSpeed=2000,memory=4096M (ensure 4 times over previous memory size), then you will encounter scaling failed
      6. Stop VM instance , you will never startup again

      EXPECTED BEHAVIOR
      ==================
      Succeed Startup VM instance

      ACTUAL BEHAVIOR
      ==================
      Fail to start instance

      Attachments

        Issue Links

          Activity

            People

              sudhansu Sudhansu Sahu
              sudhansu Sudhansu Sahu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: