Uploaded image for project: 'CloudStack'
  1. CloudStack
  2. CLOUDSTACK-2311

Live migrating VMs fails

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not A Problem
    • 4.0.2
    • None
    • None
    • Security Level: Public (Anyone can view this level - this is the default.)
    • RHEL6 + KVM

    Description

      Attempting to live migrate a VM fails. It appears to be instigating the migration process on the target host rather than the source host, attempting to stop the VM on the target host where it obviously does not yet exist. This triggers an error and the process failes. These logs from the TARGET host (note, no logs related to the migration appear on the source host):

      2013-05-03 08:43:45,421 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Request:Seq 1-1719207486: { Cmd , MgmtId: 345043945019, via: 1, Ver: v1, Flags: 100111, [{"PrepareForMigrationCommand":{"vm":{"id":2113,"name":"i-2-2113-VM","type":"User","cpus":1,"speed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Red Hat Enterprise Linux 6.0 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"920d2ec945bfef09","params":{},"uuid":"432e2eb7-e6f2-4ef7-8d4f-a2e4f3a53ab6","disks":[

      {"id":2165,"name":"ROOT-2113","mountPoint":"/labcloud_primary","path":"cadfa6a0-64ba-4f1d-a16b-cc44261e0bce","size":5368709120,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"c839c678-d2f0-3340-b379-cd24a4a73151","deviceId":0}

      ,

      {"id":2166,"name":"DATA-2113","mountPoint":"/labcloud_primary","path":"490b6738-6713-48bd-ae97-c9b6b9b2c092","size":21474836480,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"c839c678-d2f0-3340-b379-cd24a4a73151","deviceId":1}

      ],"nics":[

      {"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"6561d394-3a03-4a9e-bb31-b2c6ab24fa9d","ip":"192.168.0.205","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"02:00:77:99:00:0b","dns1":"211.29.132.12","dns2":"198.142.0.51","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2280","isolationUri":"vlan://2280","isSecurityGroupEnabled":false,"name":"cloudbr0"}

      ]},"wait":0}}] }
      2013-05-03 08:43:45,422 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Processing command: com.cloud.agent.api.PrepareForMigrationCommand
      2013-05-03 08:43:45,422 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null) Preparing host for migrating com.cloud.agent.api.to.VirtualMachineTO@782be20e
      2013-05-03 08:43:45,444 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) nic=[Nic:Guest-192.168.0.205-vlan://2280]
      2013-05-03 08:43:45,444 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) Executing: /usr/lib64/cloud/common/scripts/vm/network/vnet/modifyvlan.sh -v 2280 -p eth1 -o add
      2013-05-03 08:43:45,487 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) Execution is successful.
      2013-05-03 08:43:45,488 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-4:null) Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config

      2013-05-03 08:43:45,491 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Seq 1-1719207486: { Ans: , MgmtId: 345043945019, via: 1, Ver: v1, Flags: 110, [{"PrepareForMigrationAnswer":{"result":true,"wait":0}}] }
      2013-05-03 08:43:45,685 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) Request:Seq 1-1719207487: { Cmd , MgmtId: 345043945019, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-2-2113-VM","wait":0}}] }
      2013-05-03 08:43:45,685 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) Processing command: com.cloud.agent.api.StopCommand
      2013-05-03 08:43:45,725 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      2013-05-03 08:43:45,743 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      2013-05-03 08:43:45,744 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Executing: /usr/lib64/cloud/common/scripts/vm/network/security_group.py destroy_network_rules_for_vm --vmname i-2-2113-VM
      2013-05-03 08:43:45,993 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Execution is successful.
      2013-05-03 08:43:45,994 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Try to stop the vm at first
      2013-05-03 08:43:46,012 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to stop VM :i-2-2113-VM :
      org.libvirt.LibvirtException: Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      at org.libvirt.ErrorHandler.processError(Unknown Source)
      at org.libvirt.Connect.processError(Unknown Source)
      at org.libvirt.Connect.domainLookupByUUIDString(Unknown Source)
      at org.libvirt.Connect.domainLookupByUUID(Unknown Source)
      at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3804)
      at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:3736)
      at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2664)
      at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:991)
      at com.cloud.agent.Agent.processRequest(Agent.java:518)
      at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:831)
      at com.cloud.utils.nio.Task.run(Task.java:83)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      at java.lang.Thread.run(Thread.java:662)
      2013-05-03 08:43:46,034 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      2013-05-03 08:43:46,052 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      2013-05-03 08:43:46,071 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'
      2013-05-03 08:43:46,071 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Can't get vm's status, assume it's dead already
      2013-05-03 08:43:46,089 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching uuid '9d7e8ec6-2d11-3c65-a7a0-9e5242c5a635'

      Thanks,
      Chris

      Attachments

        Activity

          People

            Unassigned Unassigned
            cportman Chris Portman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: