When a host is put into maintenance, CloudStack schedules migration for all the running VMs present on the host. This scheduling is managed by High Availability worker thread. And every time a migration fails, CloudStack re-schedules the migration to be executed after 10 minutes.
In this case, CloudStack fails to migrate some VMs automatically while host is preparing for maintenance and admin tried to migrate them manually. All these VMs are turned off after manual migration.
- Put a host into maintenance
- Scheduled migration failed for a VM and CloudStack re-scheduled it.
- Before the next scheduled migration, manually migrate the VM to a different host.
When the next scheduled migration was started by the HA work, it failed because there was a mismatch between the source host saved in the HA work job and the actual source host. If VM migration fails due to mismatch then the VM is stopped on the host it resides on.