[CLOUDSTACK-2985] Scaling up cpu is not triggering live migration

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 4.2.0
• Fix Version/s: 4.2.0
• Component/s: Management Server
• Security Level: Public (Anyone can view this level - this is the default.)
• Environment: VMware esxi 5.1
• Assignee: Sateesh Chodapuneedi
• Reporter: prashant kumar mishra
• Created: 13/Jun/13 11:28
• Updated: 13/Aug/13 05:05
• Resolved: 12/Aug/13 23:37

Description

Steps to reproduce
-----------------------------
1-have a vmware cluster with one host
2-exhaust cpu capacity of cluster by deploying vms
3-Add one more host to cluster
4-try to scale up a vm from SO ( RAM :512,CPU:500 MHz) to SO (RAM:512 to CPU:1000)

Expected
----------------
Since there is no cpu resource left on host , vm should scale up after live migration

Actual
---------
VM scale up failed due to "The available CPU resources in the parent resource pool are insufficient for the operation"

Snippet of Log
------------------------
2013-06-13 12:42:04,802 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) RAM STATS after allocation: for host: 1, old used: 5502926848, old reserved: 0, total: 17169539072; new used: 6039797760, reserved: 0; requested mem: 536870912,alloc_from_last:false
2013-06-13 12:42:04,813 DEBUG [agent.transport.Request] (Job-Executor-19:job-288) Seq 1-974783027: Sending { Cmd , MgmtId: 7635042566263, via: 1, Ver: v1, Flags: 100111, [{"ScaleVmCommand":{"vm":

{"id":1,"name":"i-2-45-VM","cpus":1,"speed":1000,"minRam":512,"maxRam":512,"rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false}

,"vmName":"i-2-45-VM","cpus":1,"speed":1000,"minRam":512,"maxRam":512,"wait":0}}] }
2013-06-13 12:42:04,815 DEBUG [agent.transport.Request] (Job-Executor-19:job-288) Seq 1-974783027: Executing: { Cmd , MgmtId: 7635042566263, via: 1, Ver: v1, Flags: 100111, [{"ScaleVmCommand":{"vm":

{"id":1,"name":"i-2-45-VM","cpus":1,"speed":1000,"minRam":512,"maxRam":512,"rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false}

,"vmName":"i-2-45-VM","cpus":1,"speed":1000,"minRam":512,"maxRam":512,"wait":0}}] }
2013-06-13 12:42:04,815 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-213:null) Seq 1-974783027: Executing request
2013-06-13 12:42:04,816 DEBUG [vmware.mo.HostMO] (DirectAgent-213:10.147.40.7) find VM i-2-45-VM on host
2013-06-13 12:42:04,816 DEBUG [vmware.mo.HostMO] (DirectAgent-213:10.147.40.7) load VM cache on host
2013-06-13 12:42:06,191 ERROR [vmware.resource.VmwareResource] (DirectAgent-213:10.147.40.7) Unexpected exception:
java.lang.RuntimeException: The available CPU resources in the parent resource pool are insufficient for the operation.
at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:291)
at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.configureVm(VirtualMachineMO.java:832)
at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2331)
at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:559)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-06-13 12:42:06,193 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-213:null) Seq 1-974783027: Response Received:
2013-06-13 12:42:06,194 DEBUG [agent.transport.Request] (DirectAgent-213:null) Seq 1-974783027: Processing: { Ans: , MgmtId: 7635042566263, via: 1, Ver: v1, Flags: 110, [{"ScaleVmAnswer":{"result":false,"details":"Unable to execute ScaleVmCommand due to java.lang.RuntimeException: The available CPU resources in the parent resource pool are insufficient for the operation.","wait":0}}] }
2013-06-13 12:42:06,194 DEBUG [agent.transport.Request] (Job-Executor-19:job-288) Seq 1-974783027: Received: { Ans: , MgmtId: 7635042566263, via: 1, Ver: v1, Flags: 110,

{ ScaleVmAnswer }

}
2013-06-13 12:42:06,194 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-19:job-288) Unable to scale vm due to Unable to execute ScaleVmCommand due to java.lang.RuntimeException: The available CPU resources in the parent resource pool are insufficient for the operation.
2013-06-13 12:42:06,199 DEBUG [agent.manager.AgentAttache] (DirectAgent-213:null) Seq 1-974783027: No more commands found
2013-06-13 12:42:06,209 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) Hosts's actual total CPU: 9572 and CPU after applying overprovisioning: 9572
2013-06-13 12:42:06,209 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) Hosts's actual total RAM: 17169539072 and RAM after applying overprovisioning: 17169539072
2013-06-13 12:42:06,209 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) release cpu from host: 1, old used: 8200,reserved: 0, actual total: 9572, total with overprovisioning: 9572; new used: 7200,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-06-13 12:42:06,209 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) release mem from host: 1, old used: 6039797760,reserved: 0, total: 17169539072; new used: 5502926848,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-06-13 12:42:06,233 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) Hosts's actual total CPU: 9572 and CPU after applying overprovisioning: 9572
2013-06-13 12:42:06,233 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) We are allocating VM, increasing the used capacity of this host:1
2013-06-13 12:42:06,233 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) Current Used CPU: 7200 , Free CPU:2372 ,Requested CPU: 500
2013-06-13 12:42:06,233 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) Current Used RAM: 5502926848 , Free RAM:11666612224 ,Requested RAM: 536870912
2013-06-13 12:42:06,233 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) CPU STATS after allocation: for host: 1, old used: 7200, old reserved: 0, actual total: 9572, total with overprovisioning: 9572; new used:7700, reserved:0; requested cpu:500,alloc_from_last:false
2013-06-13 12:42:06,234 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-19:job-288) RAM STATS after allocation: for host: 1, old used: 5502926848, old reserved: 0, total: 17169539072; new used: 6039797760, reserved: 0; requested mem: 536870912,alloc_from_last:false
2013-06-13 12:42:06,264 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-19:job-288) Unexpected exception while executing org.apache.cloudstack.api.command.user.vm.ScaleVMCmd
com.cloud.utils.exception.CloudRuntimeException: Unable to scale vm due to Unable to execute ScaleVmCommand due to java.lang.RuntimeException: The available CPU resources in the parent resource pool are insufficient for the operation.
at com.cloud.vm.VirtualMachineManagerImpl.reConfigureVm(VirtualMachineManagerImpl.java:3250)
at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1227)
at com.cloud.vm.UserVmManagerImpl.upgradeVirtualMachine(UserVmManagerImpl.java:1106)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at org.apache.cloudstack.api.command.user.vm.ScaleVMCmd.execute(ScaleVMCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-06-13 12:42:06,266 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-19:job-288) Complete async job-288, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Unable to scale vm due to Unable to execute ScaleVmCommand due to java.lang.RuntimeException: The available CPU resources in the parent resource pool are insufficient for the operation.

Issue Links

• is superceded by: CLOUDSTACK-1809 Actual capacity availble for vm deployement in vmware cluster is less than what Dashboard shows (total_capacity in op_host_capacity table)
• requires: CLOUDSTACK-1809 Actual capacity availble for vm deployement in vmware cluster is less than what Dashboard shows (total_capacity in op_host_capacity table)