Unable to delete openstack instance stuck at Error state
When you attempt to delete instance from either openstack CLI or openstack dashboard and if it get stuck at error state as follows:
GUI:
CLI:
`nova list` will show:
| 1541a197-
There may be so many reasons for this error state like attached volume may not able to reclaim, or unable to reclaim associated port by neutron and so on.
Sometimes we can try from CLI after resetting state “nova reset-state <instance>” using “nova delete $instanceid”
When above workaround is not working along with any other options, at last if you want to mark instance as deleted in nova database then we can follow below approach:
Note: Please use this when you hit the wall other wise not recommended:
Step 1: Get instance details using “nova show “$instanceid””
[root@controller ~]# nova show 92b9c565-d908-4dd0-a169-f58b3ce71428
| Property | Value |
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | kvm-02 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | kvm-02 |
| OS-EXT-SRV-ATTR:instance_name | instance-0000005b |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | – |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-05-16T09:35:14.000000 |
| OS-SRV-USG:terminated_at | – |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2016-05-16T09:35:09Z |
| flavor | Standard 5 (b6be6358-120a-4fb9-987c-f10a7e82cb09) |
| hostId | b8b17a365b15d16f31860029f971348e8c8af7f2092652c5e7c36b83 |
| id | 92b9c565-d908-4dd0-a169-f58b3ce71428 |
| image | CentOS-7-x86_64-GenericCloud-1603.qcow2 (7e78fc90-4eed-41a5-8a79-9fef27371278) |
| key_name | suresh |
| metadata | {} |
| name | Jenkins |
| net-int network | 10.10.10.82, 202.65.153.24 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | 87725975eafe46ebb8c54e02acf0b654 |
| updated | 2016-05-16T09:36:04Z |
| user_id | 670959f49dde4aa582d99afbf6835387 |
Step 2: Connect to mysql database
[root@controller ~]# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or g.
Your MariaDB connection id is 241348
Server version: 5.5.47-MariaDB MariaDB Server
Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.
Type ‘help;’ or ‘h’ for help. Type ‘c’ to clear the current input statement.
MariaDB [(none)]> show databases;
+——————–+
| Database |
+——————–+
| information_schema |
| cinder |
| glance |
| keystone |
| mysql |
| neutron |
| nova |
| performance_schema |
| test |
+——————–+
9 rows in set (0.00 sec)
#Now change the database to use nova:
MariaDB [(none)]> use nova;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [nova]>
Step 3: Get the current status of instance from database
MariaDB [nova]> select uuid, hostname, vm_state, task_state, power_state, deleted from instances where uuid=”92b9c565-d908-4dd0-a169-f58b3ce71428″;
+————————————–+———-+———-+————+————-+———+
| uuid | hostname | vm_state | task_state | power_state | deleted |
+————————————–+———-+———-+————+————-+———+
| 92b9c565-d908-4dd0-a169-f58b3ce71428 | jenkins | active | NULL | 1 | 0 |
+————————————–+———-+———-+————+————-+———+
1 row in set (0.00 sec)
##Mark instance deleted state to 1 using below query
MariaDB [nova]> update instances set deleted=’1′, vm_state=’deleted’, deleted_at=’now()’ where uuid=’77beaec2-636e-4e3e-9867-5df057fc94be’ and project_id=’87725975eafe46ebb8c54e02acf0b654′;
validate the state again by running above status query:
MariaDB [nova]> select uuid, hostname, vm_state, task_state, power_state, deleted from instances where uuid=”92b9c565-d908-4dd0-a169-f58b3ce71428″;
+————————————–+———-+———-+————+————-+———+
| uuid | hostname | vm_state | task_state | power_state | deleted |
+————————————–+———-+———-+————+————-+———+
| 92b9c565-d908-4dd0-a169-f58b3ce71428 | jenkins | deleted| NULL | 0 | 1 |
+————————————–+———-+———-+————+————-+———+
1 row in set (0.00 sec)
Now you can see your power state and deleted db columns set to 0,1 respectively even this instance will be removed from openstack dashboard also.
Hope this post helped…
Nice Post !! Before getting into this we can also give a try by restarting the libvirt and nova compute service as below
Ubuntu:
service libvirt-bin restart
service nova-compute restart
RedHat:
systemctl stop openstack-nova-compute.service
systemctl restart libvirtd.service
systemctl start openstack-nova-compute.service