Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
4.6.0, 4.7.0
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
Description
This issue appears with commit https://github.com/apache/cloudstack/commit/a709f34ff93579981afbd4df0f4334b61ada29a8
where xmlDesc has some content replaced:
xmlDesc = dm.getXMLDesc(xmlFlag).replace(libvirtComputingResource.getPrivateIp(), command.getDestinationIp());
This line from LibvirtComputingResource.java was refactored into LibvirtMigrateCommandWrapper.java in commit https://github.com/apache/cloudstack/commit/28e55462f15bdd8699e97b668c4ffc01735a533d
Example, node1 is 10.0.0.1, node2 is 10.0.0.2, rbd mon_host is 10.0.0.13.
VM is running on kvm node1 and this happened when migrating a VM from node1 to node2, where the kvm nodes and the RBD mon nodes (mon_host) use IP addresses in the same range, and the mon_host has an ip address that partially matches the ip address (string) of the first kvm node.
In the process list one can see that the mon_host changes from 10.0.0.13 to 10.0.0.23 in the destination host, blocking the migration after a timeout due to primary storage not being available.
root 25206 1.8 0.0 440184 17188 ? Sl 13:33 0:00 /usr/libexec/qemu-kvm -name r-74-VM -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -cpu host,+rdtscp,+pdpe1gb,+dca,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 256 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 1a32b655-0acf-424b-8722-9e7f507a3070 -smbios type=1,manufacturer=Apache Software Foundation,product=CloudStack KVM Hypervisor,uuid=1a32b655-0acf-424b-8722-9e7f507a3070 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-r-74-VM/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=rbd:rbd/15ea00f9-e52e-43cf-9e5e-62188e9da5d2:id=cloudstack:key=XXXXXXXX==:auth_supported=cephx\;none:mon_host=10.0.0.23\:6789,if=none,id=drive-virtio-disk0,format=raw,serial=15ea00f9e52e43cf9e5e,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/usr/share/cloudstack-common/vms/systemvm.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=32,id=hostnet0,vhost=on,vhostfd=33 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=11:11:11:11:11:11,bus=pci.0,addr=0x3,rombar=0,romfile= -netdev tap,fd=34,id=hostnet1,vhost=on,vhostfd=35 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=22:22:22:22:22:22,bus=pci.0,addr=0x4,rombar=0,romfile= -netdev tap,fd=36,id=hostnet2,vhost=on,vhostfd=37 -device virtio-net-pci,netdev=hostnet2,id=net2,mac=33:33:33:33:33:33,bus=pci.0,addr=0x5,rombar=0,romfile= -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/r-74-VM.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=r-74-VM.vport -device usb-tablet,id=input0 -vnc 10.0.0.2:4,password -vga cirrus -incoming tcp:[::]:49152 -msg timestamp=on