Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
4.6.0
-
None
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
Description
Attempting to remove a disabled zone fails.
Before disabling the zone, all instances were destroyed and the systemvms were stopped. Then the agent was stopped, the vms destroyed in the host and the host was removed from the management server.
Trying to delete the zone in UI and cloudmonkey returns the following:
(local) 🐵 > delete zone id=20974fbb-738e-4915-95f1-acd1f437c35b
Error 530: The zone cannot be deleted because there are storage volumes in this zone.
Went to the database and started doing cleanup.
There were multiple entries because I was testing configurations and had to add the same host repeatedly:
MariaDB [cloud]> select * from host; +----+----------------------------------+--------------------------------------+---------+---------+--------------------+-----------------+---------------------+--------------------+-----------------+---------------------+----------------------+-----------------------+-------------------+------------+-------------------+----------------+--------------------+------------+----------------+--------+-------------+------+-------+------+---------+-----------------+--------------------+--------------+----------+---------+--------+------------+--------------+---------------------------------------------------------------+-----------+-------+-------------+------------+-----------------+---------------------+---------------------+---------------------+--------------+----------------+-------+-------------+--------------+ | id | name | uuid | status | type | private_ip_address | private_netmask | private_mac_address | storage_ip_address | storage_netmask | storage_mac_address | storage_ip_address_2 | storage_mac_address_2 | storage_netmask_2 | cluster_id | public_ip_address | public_netmask | public_mac_address | proxy_port | data_center_id | pod_id | cpu_sockets | cpus | speed | url | fs_type | hypervisor_type | hypervisor_version | ram | resource | version | parent | total_size | capabilities | guid | available | setup | dom0_memory | last_ping | mgmt_server_id | disconnected | created | removed | update_count | resource_state | owner | lastUpdated | engine_state | +----+----------------------------------+--------------------------------------+---------+---------+--------------------+-----------------+---------------------+--------------------+-----------------+---------------------+----------------------+-----------------------+-------------------+------------+-------------------+----------------+--------------------+------------+----------------+--------+-------------+------+-------+------+---------+-----------------+--------------------+--------------+----------+---------+--------+------------+--------------+---------------------------------------------------------------+-----------+-------+-------------+------------+-----------------+---------------------+---------------------+---------------------+--------------+----------------+-------+-------------+--------------+ | 3 | node01 | a191501a-2a09-49d2-b713-cb1b86978759 | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 54 | 9 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414521468 | NULL | NULL | 2015-11-25 16:44:31 | 2015-11-25 16:46:27 | 43 | Enabled | NULL | NULL | Disabled | | 6 | node01 | 11fd7da4-f251-49a1-93be-5e5fe6a439ad | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 54 | 9 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414521509 | NULL | NULL | 2015-11-25 16:46:28 | 2015-11-25 16:47:08 | 17 | Disabled | NULL | NULL | Disabled | | 9 | node01 | 453afe7d-6945-46d3-92f5-a48dc8a3627a | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 54 | 9 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414521570 | NULL | NULL | 2015-11-25 16:47:10 | 2015-11-25 16:48:08 | 25 | Disabled | NULL | NULL | Disabled | | 12 | node01 | a0989225-8494-4c2a-806e-d2aac625cbce | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 54 | 9 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414521585 | NULL | NULL | 2015-11-25 16:48:13 | 2015-11-25 16:48:36 | 7 | Enabled | NULL | NULL | Disabled | | 15 | node01 | aed1524b-ba91-497f-887a-f3beba53dcb0 | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 54 | 9 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414522634 | NULL | NULL | 2015-11-25 16:53:47 | 2015-11-25 17:06:37 | 261 | Enabled | NULL | NULL | Disabled | | 18 | node01 | 59125ab9-74b0-4658-8ca7-06f534a3b951 | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 57 | 12 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414578945 | NULL | NULL | 2015-11-25 17:17:12 | 2015-11-26 10:32:43 | 16 | Enabled | NULL | NULL | Disabled | | 21 | node01 | b8a01b59-d747-4e96-a976-f60d8bec2c1d | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 60 | 15 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414586121 | NULL | NULL | 2015-11-26 11:08:30 | 2015-11-26 11:10:21 | 33 | Enabled | NULL | NULL | Disabled | | 24 | node01 | fc8b1f3d-007f-4cfd-88b8-d54939c37f71 | Removed | Routing | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | 10.0.0.1 | 255.255.255.0 | ff:ff:ff:ff:ff:ff | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 60 | 15 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | NULL | 1 | 0 | 0 | 1414594868 | NULL | NULL | 2015-11-26 11:15:20 | 2015-11-26 13:39:59 | 105 | Enabled | NULL | NULL | Disabled | | 27 | node01 | 7b3626b6-8c9e-406c-afb0-2f8ee47d2647 | Up | Routing | 10.0.1.1 | 255.255.255.0 | aa:aa:aa:aa:aa:aa | 10.0.1.1 | 255.255.255.0 | aa:aa:aa:aa:aa:aa | NULL | NULL | NULL | 21 | NULL | NULL | NULL | NULL | 63 | 18 | 1 | 24 | 2394 | NULL | NULL | KVM | NULL | 201583407104 | NULL | 4.6.0 | NULL | NULL | hvm,snapshot | bb3e4993-ea79-304e-a74c-9499238276d5-LibvirtComputingResource | 1 | 0 | 0 | 1414597657 | 233845177548613 | 2015-11-26 14:26:29 | 2015-11-26 13:57:46 | NULL | 7 | Enabled | NULL | NULL | Disabled | +----+----------------------------------+--------------------------------------+---------+---------+--------------------+-----------------+---------------------+--------------------+-----------------+---------------------+----------------------+-----------------------+-------------------+------------+-------------------+----------------+--------------------+------------+----------------+--------+-------------+------+-------+------+---------+-----------------+--------------------+--------------+----------+---------+--------+------------+--------------+---------------------------------------------------------------+-----------+-------+-------------+------------+-----------------+---------------------+---------------------+---------------------+--------------+----------------+-------+-------------+--------------+ 9 rows in set (0.00 sec) MariaDB [cloud]> select id,removed from host; +----+---------------------+ | id | removed | +----+---------------------+ | 27 | NULL | | 3 | 2015-11-25 16:46:27 | | 6 | 2015-11-25 16:47:08 | | 9 | 2015-11-25 16:48:08 | | 12 | 2015-11-25 16:48:36 | | 15 | 2015-11-25 17:06:37 | | 18 | 2015-11-26 10:32:43 | | 21 | 2015-11-26 11:10:21 | | 24 | 2015-11-26 13:39:59 | +----+---------------------+ 9 rows in set (0.00 sec)
Removing the removed hosts from the table:
MariaDB [cloud]> delete from host where id=3; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=6; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=9; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=12; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=15; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=18; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=21; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=24; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`cloud`.`op_ha_work`, CONSTRAINT `fk_op_ha_work__host_id` FOREIGN KEY (`host_id`) REFERENCES `host` (`id`))
There is a host that cannot be removed due to a foreign key constraint, following the bread crumbs:
MariaDB [cloud]> select * from op_ha_work; +----+-------------+-----------+--------------------+---------+-----------------+---------+---------------------+-------+---------------------+------+-------------+---------+ | id | instance_id | type | vm_type | state | mgmt_server_id | host_id | created | tried | taken | step | time_to_try | updated | +----+-------------+-----------+--------------------+---------+-----------------+---------+---------------------+-------+---------------------+------+-------------+---------+ | 3 | 1 | ForceStop | ConsoleProxy | Running | 233845177548613 | 24 | 2015-11-26 13:23:32 | 0 | 2015-11-26 13:23:32 | Done | 1414593957 | 3 | | 6 | 2 | ForceStop | SecondaryStorageVm | Running | 233845177548613 | 24 | 2015-11-26 13:23:32 | 0 | 2015-11-26 13:23:32 | Done | 1414593957 | 3 | +----+-------------+-----------+--------------------+---------+-----------------+---------+---------------------+-------+---------------------+------+-------------+---------+ 2 rows in set (0.00 sec) MariaDB [cloud]> delete from op_ha_work where id=3; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from op_ha_work where id=6; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from host where id=24; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`cloud`.`vm_instance`, CONSTRAINT `fk_vm_instance__last_host_id` FOREIGN KEY (`last_host_id`) REFERENCES `host` (`id`))
The rabbit hole is deeper than expected, following more bread crumbs:
MariaDB [cloud]> select * from vm_instance; +----+--------+--------------------------------------+---------------+-----------+----------------+-------------+---------------------+--------------------+--------+----------------+---------+--------------+----------+---------------------+----------------------------------------------+------------+---------------+--------------+---------------------+---------------------+---------------------+--------------------+--------------------+------------+-----------+---------------------+--------------------------------------+-----------------+------------------+-------+-----------+--------------+---------------+----------------------+------------+-------------+-------------------------+--------------------------+------------+---------+ | id | name | uuid | instance_name | state | vm_template_id | guest_os_id | private_mac_address | private_ip_address | pod_id | data_center_id | host_id | last_host_id | proxy_id | proxy_assign_time | vnc_password | ha_enabled | limit_cpu_use | update_count | update_time | created | removed | type | vm_type | account_id | domain_id | service_offering_id | reservation_id | hypervisor_type | disk_offering_id | owner | host_name | display_name | desired_state | dynamically_scalable | display_vm | power_state | power_state_update_time | power_state_update_count | power_host | user_id | +----+--------+--------------------------------------+---------------+-----------+----------------+-------------+---------------------+--------------------+--------+----------------+---------+--------------+----------+---------------------+----------------------------------------------+------------+---------------+--------------+---------------------+---------------------+---------------------+--------------------+--------------------+------------+-----------+---------------------+--------------------------------------+-----------------+------------------+-------+-----------+--------------+---------------+----------------------+------------+-------------+-------------------------+--------------------------+------------+---------+ | 1 | v-1-VM | 4e48d5c9-193c-4ace-b06f-f045eb40b7e7 | v-1-VM | Destroyed | 3 | 15 | NULL | NULL | 15 | 60 | NULL | 24 | 1 | 2015-11-26 11:35:11 | mRpIU2IviYOIvfySyb4474nkHaqn4DJGPPChAUNM09A= | 0 | 0 | 8 | 2015-11-26 13:23:44 | 2015-11-26 11:33:24 | 2015-11-26 13:23:44 | ConsoleProxy | ConsoleProxy | 1 | 1 | 27 | 8c993bcc-ba0d-4ff8-9e15-525a22222cbe | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | PowerOn | 2015-11-26 11:36:41 | 3 | 24 | 1 | | 2 | s-2-VM | bd7c05e3-616b-457f-bb5e-678d324d8853 | s-2-VM | Destroyed | 3 | 15 | 06:cc:76:00:01:78 | NULL | 15 | 60 | NULL | 24 | 1 | 2015-11-26 11:35:38 | 58oA9Z0qUJY80vHzbXwvzj3VHNypAyHMJ12p9JVNC0g= | 0 | 0 | 8 | 2015-11-26 13:23:44 | 2015-11-26 11:33:29 | 2015-11-26 13:23:44 | SecondaryStorageVm | SecondaryStorageVm | 1 | 1 | 21 | 1f5b5a2c-7df9-4ab1-b040-3dfea10cf0a7 | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | PowerOn | 2015-11-26 11:37:41 | 3 | 24 | 1 | | 3 | v-3-VM | fc404731-a903-414a-abc1-777fe559115d | v-3-VM | Destroyed | 3 | 15 | NULL | NULL | 15 | 60 | NULL | 24 | NULL | NULL | HytvsL4jeniDaBfZPlMPGqXq6hXKnCGApMAIwHFiSdE= | 0 | 0 | 6 | 2015-11-26 14:00:47 | 2015-11-26 13:23:46 | 2015-11-26 14:00:47 | ConsoleProxy | ConsoleProxy | 1 | 1 | 27 | 0bde1391-d4ed-4b0a-91e8-adfda46cf2b2 | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | NULL | NULL | 0 | NULL | 1 | | 4 | s-4-VM | 133cb5e5-7d9b-4320-993a-94824e716c46 | s-4-VM | Destroyed | 3 | 15 | 06:31:64:00:01:79 | NULL | 15 | 60 | NULL | 24 | NULL | NULL | 42rU/ytVMQdTvG5DAaQsOoBBVet+tPLV2EvzY1vgeiw= | 0 | 0 | 6 | 2015-11-26 13:59:49 | 2015-11-26 13:23:46 | 2015-11-26 13:59:49 | SecondaryStorageVm | SecondaryStorageVm | 1 | 1 | 21 | df386544-13f8-43e2-9b46-0276a83125b5 | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | NULL | NULL | 0 | NULL | 1 | | 5 | r-5-VM | c8279539-25f5-4686-9a57-cb9bfd96a93e | r-5-VM | Destroyed | 3 | 15 | 0e:00:a9:fe:00:80 | 169.254.0.128 | 15 | 60 | NULL | 24 | NULL | NULL | BmM552nDQtQS7g8UI/Apo373tLhXb8rY/nZRFwv7d4Y= | 1 | 0 | 8 | 2015-11-26 13:33:40 | 2015-11-26 13:24:19 | 2015-11-26 13:33:41 | DomainRouter | DomainRouter | 1 | 1 | 39 | d1aae1d1-4d4a-41a8-930e-42a427a0bd56 | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | NULL | NULL | 0 | NULL | 1 | | 6 | r-6-VM | 911f6cf4-0888-45e3-bac7-3bcb3a8c2fd3 | r-6-VM | Destroyed | 3 | 15 | 0e:00:a9:fe:02:f8 | 169.254.2.248 | 15 | 60 | NULL | 24 | NULL | NULL | GN3eCogo8wjZYS4FaRDOYvlK73etPlc6lqHcXPLNS3w= | 1 | 0 | 6 | 2015-11-26 13:40:11 | 2015-11-26 13:31:30 | 2015-11-26 13:40:11 | DomainRouter | DomainRouter | 1 | 1 | 39 | 50edc070-6099-4c9f-ac04-06b8e6452faf | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | NULL | NULL | 0 | NULL | 1 | | 7 | r-7-VM | 107219e3-f1f8-43a0-bcba-56bb9c33bd94 | r-7-VM | Destroyed | 3 | 15 | 0e:00:a9:fe:03:66 | 169.254.3.102 | 15 | 60 | NULL | 24 | NULL | NULL | OnmMub3WRY25T8IX10SzAgQZ17YJCyw8UaV2oUSV+bE= | 1 | 0 | 6 | 2015-11-26 13:40:11 | 2015-11-26 13:33:41 | 2015-11-26 13:40:11 | DomainRouter | DomainRouter | 1 | 1 | 39 | c3fd4753-ca8f-4b9a-8d26-8e1bda3c0c18 | KVM | NULL | NULL | NULL | NULL | NULL | 0 | 1 | NULL | NULL | 0 | NULL | 1 | +----+--------+--------------------------------------+---------------+-----------+----------------+-------------+---------------------+--------------------+--------+----------------+---------+--------------+----------+---------------------+----------------------------------------------+------------+---------------+--------------+---------------------+---------------------+---------------------+--------------------+--------------------+------------+-----------+---------------------+--------------------------------------+-----------------+------------------+-------+-----------+--------------+---------------+----------------------+------------+-------------+-------------------------+--------------------------+------------+---------+ 7 rows in set (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=1; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=2; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=3; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=4; Query OK, 1 row affected (0.01 sec) MariaDB [cloud]> delete from vm_instance where id=5; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=6; Query OK, 1 row affected (0.00 sec) MariaDB [cloud]> delete from vm_instance where id=7; Query OK, 1 row affected (0.00 sec)
That should do it:
MariaDB [cloud]> delete from host where id=24; Query OK, 1 row affected (0.00 sec)
The zone was able to be removed after the cleanup.