[CLOUDSTACK-4826] System VMs fail to start

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.2.0
• Fix Version/s: 4.3.0
• Component/s: Doc, KVM
• Security Level: Public (Anyone can view this level - this is the default.)

Environment

CentOS 6.4
qemu-kvm-0.12.1.2-2.355.0.1.el6_4.9.x86_64

Cloudstack installed from RPM repo listed in docs

AND:

libvirt-client-0.10.2-18.el6_4.14.x86_64
libvirt-0.10.2-18.el6_4.14.x86_64

qemu-kvm-0.12.1.2-2.355.0.1.el6.centos.7.x86_64

CloudStack 4.2 upgraded based on 4.2 release guide steps

• Assignee: Kishan
• Reporter: Dave Garbus
• Created: 08/Oct/13 05:37
• Updated: 24/Nov/21 10:15
• Resolved: 31/Dec/13 11:14

Description

After upgrading from 4.1.1 to 4.2, system VMs did not restart properly when running cloudstack-sysvmadm. Since we do not rely heavily on them at this point, I removed them figuring they would simply be recreated (this has worked in the past).

When CloudStack attempts to recreate the VMs, provisioning fails:

agent.log (IPs are obscured):
========================
Timed out: /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.pl -n s-111-VM -p %template=domP%type=secstorage%host=XXXX.com%port=8250%name=s-111-VM%zone=1%pod=1%guid=s-111-VM%resource=com.cloud.storage.resource.PremiumSecondaryStorageResource%instance=SecStorage%sslcopy=true%role=templateProcessor%mtu=1500%eth2ip=XX.XX.XX.XX%eth2mask=255.255.255.0%gateway=XX.XX.XX.XX%public.network.device=eth2%eth0ip=169.254.0.47%eth0mask=255.255.0.0%eth1ip=XX.XX.XX.XX%eth1mask=255.255.255.0%mgmtcidr=XX.XX.XX.0/29%localgw=XX.XX.XX.XX%private.network.device=eth1%eth3ip=XX.XX.XX.XX%eth3mask=255.255.255.0%storageip=XX.XX.XX.XX%storagenetmask=255.255.255.0%storagegateway=XX.XX.XX.XX%internaldns1=XX.XX.XX.XX%internaldns2=XX.XX.XX.XX%dns1=XX.XX.XX.XX%dns2=XX.XX.XX.XX . Output is:

I gained access to the VM using the root password, and this is what I found:

root@systemvm:~# cat /etc/cloudstack-release
Cloudstack Release 3.0 Mon Feb 6 15:10:04 PST 2012
root@systemvm:~# uname -a
Linux systemvm 2.6.32-5-686-bigmem #1 SMP Mon Jan 16 16:42:05 UTC 2012 i686 GNU/Linux
root@systemvm:~# /etc/init.d/cloud-
cloud-early-config cloud-passwd-srvr
root@systemvm:~# /etc/init.d/cloud-early-config start
Executing cloud-early-config...Executing cloud-early-config...Detected that we are running inside kvm guest.../dev/vport0p1 not loaded, perhaps guest kernel is too old....root@systemvm:~#

I have the system vm template systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2, which is the latest (to my knowledge), on my secondary storage NFS mount, however, the SSVM is not able to be started, so I'm not sure this helps.

Please let me know if more information is needed.