So I uploaded a qcow image…but did it wrong. It was tagged as raw instead of qcow, and now I want it gone. Only problem….it is stuck.
$ openstack image delete rhel-server-7.4-update-4-x86_64 Failed to delete image with name or ID 'rhel-server-7.4-update-4-x86_64': 409 Conflict Image 2e77971e-7746-4992-8e1e-7ce1be8528f8 could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance. |
But….I deleted all of the instances connected to it! Come On!
Answer is easy once the code-rage wears off…
When I created a server based on this image, it created a new volume. That volume is locking the image into place.
$ openstack volume list +--------------------------------------+------+-----------+------+----------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+------+-----------+------+----------------------------------+ | 97a15e9c-2744-4f31-95f3-a13603e49b6d | | error | 1 | | | c9337612-8317-425f-b313-f8ba9336f1cc | | available | 1 | | | 9560a18f-bfeb-4964-9785-6e76fa720892 | | in-use | 9 | Attached to showoff on /dev/vda | | 0188edd7-7e91-4a80-a764-50d47bba9978 | | in-use | 9 | Attached to test1 on /dev/vda | +--------------------------------------+------+-----------+------+----------------------------------+ |
See that error? I think its that one. I can’t confirm now, as I also deleted the available one, as I didn’t need it, either.
$ openstack volume delete 97a15e9c-2744-4f31-95f3-a13603e49b6d $ openstack volume delete c9337612-8317-425f-b313-f8ba9336f1cc $ openstack image delete rhel-server-7.4-update-4-x86_64 |
And that last command succeeded.
$ openstack image show rhel-server-7.4-update-4-x86_64 Could not find resource rhel-server-7.4-update-4-x86_64 |
Logon to Controller
[heat-admin@overcloud-controller-0 ~]$ source overcloudrc
List the images from CEPH node
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd ls -l images
NAME SIZE PARENT FMT PROT LOCK
8b0f9c33-44e7-44eb-8a06-87999dbdeffd 433MiB 2
8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap 433MiB 2 yes
e7b16aa1-11c7-42ec-81f6-37c49b64e108 50GiB 2
e7b16aa1-11c7-42ec-81f6-37c49b64e108@snap 50GiB 2 yes
f3e582b9-42a2-44a1-96fa-ca9b5fa3697e 12.1MiB 2
f3e582b9-42a2-44a1-96fa-ca9b5fa3697e@snap 12.1MiB 2 yes
LIST images from Openstack DB
(overcloud) [heat-admin@overcloud-controller-0 ~]$ openstack image list
+————————————–+————–+——–+
| ID | Name | Status |
+————————————–+————–+——–+
| 8b0f9c33-44e7-44eb-8a06-87999dbdeffd | Ubuntu | active |
| f3e582b9-42a2-44a1-96fa-ca9b5fa3697e | cirros-image | active |
| e7b16aa1-11c7-42ec-81f6-37c49b64e108 | snap-test001 | active |
+————————————–+————–+——–+
Before deleting – view the image details
(overcloud) [heat-admin@overcloud-controller-0 ~]$ openstack image show 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
+——————+—————————————————————————————————————————————————————————————————————————————————+
| Field | Value |
+——————+—————————————————————————————————————————————————————————————————————————————————+
| checksum | 4751a8a40971afa0fe647d2d324aff27 |
| container_format | bare |
| created_at | 2019-11-25T08:30:16Z |
| disk_format | raw |
| file | /v2/images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd/file |
| id | 8b0f9c33-44e7-44eb-8a06-87999dbdeffd |
| min_disk | 0 |
| min_ram | 0 |
| name | Ubuntu |
| owner | 4de3949edce3432bb72ccf7dc6116ea8 |
| properties | direct_url=’rbd://9f119502-ad15-11e9-86e7-5254009e02c5/images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd/snap’, locations='[{u’url’: u’rbd://9f119502-ad15-11e9-86e7-5254009e02c5/images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd/snap’, u’metadata’: {}}]’ |
| protected | False |
| schema | /v2/schemas/image |
| size | 454098944 |
| status | active |
| tags | |
| updated_at | 2019-11-25T08:30:22Z |
| virtual_size | None |
| visibility | public |
+——————+—————————————————————————————————————————————————————————————————————————————————+
Delete image
(overcloud) [heat-admin@overcloud-controller-0 ~]$ openstack image delete 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
Failed to delete image with name or ID ‘8b0f9c33-44e7-44eb-8a06-87999dbdeffd’: 409 Conflict: Image 8b0f9c33-44e7-44eb-8a06-87999dbdeffd could not be deleted because it is in use: The image cannot be deleted because it is in use through the backend store outside of Glance. (HTTP 409)
Failed to delete 1 of 1 images.
Either the VOLUME attached with VM was deleted during VM deletion, follow the below process…
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd -p vms ls -l
NAME SIZE PARENT FMT PROT LOCK
139bede4-c625-4709-9451-bf8722e70f6d_disk 20GiB 2
43a3bded-7292-4df8-a669-86f1c21782d8_disk 20GiB 2 excl
5b777e70-2d9e-4cf6-bc88-46f39add7933_disk 50GiB images/e7b16aa1-11c7-42ec-81f6-37c49b64e108@snap 2 excl
a92e61a3-e9ee-4467-b44e-3d838692c1e7_disk 10GiB 2 excl
(overcloud) [heat-admin@overcloud-controller-0 ~]$ glance image-list
+————————————–+————–+
| ID | Name |
+————————————–+————–+
| f3e582b9-42a2-44a1-96fa-ca9b5fa3697e | cirros-image |
| e7b16aa1-11c7-42ec-81f6-37c49b64e108 | snap-test001 |
| 8b0f9c33-44e7-44eb-8a06-87999dbdeffd | Ubuntu |
+————————————–+————–+
(overcloud) [heat-admin@overcloud-controller-0 ~]$ glance image-delete 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
Unable to delete image ‘8b0f9c33-44e7-44eb-8a06-87999dbdeffd’ because it is in use.
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd ls -l images
NAME SIZE PARENT FMT PROT LOCK
8b0f9c33-44e7-44eb-8a06-87999dbdeffd 433MiB 2
8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap 433MiB 2 yes
e7b16aa1-11c7-42ec-81f6-37c49b64e108 50GiB 2
e7b16aa1-11c7-42ec-81f6-37c49b64e108@snap 50GiB 2 yes
f3e582b9-42a2-44a1-96fa-ca9b5fa3697e 12.1MiB 2
f3e582b9-42a2-44a1-96fa-ca9b5fa3697e@snap 12.1MiB 2 yes
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd children images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd
rbd: snap name was not specified
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd children images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap
volumes/volume-cd2aa395-f9fc-4b26-aa7e-a34d54cd5645
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd -p vms ls -l
NAME SIZE PARENT FMT PROT LOCK
139bede4-c625-4709-9451-bf8722e70f6d_disk 20GiB 2
43a3bded-7292-4df8-a669-86f1c21782d8_disk 20GiB 2 excl
5b777e70-2d9e-4cf6-bc88-46f39add7933_disk 50GiB images/e7b16aa1-11c7-42ec-81f6-37c49b64e108@snap 2 excl
a92e61a3-e9ee-4467-b44e-3d838692c1e7_disk 10GiB 2 excl
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd –pool images info 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
rbd image ‘8b0f9c33-44e7-44eb-8a06-87999dbdeffd’:
size 433MiB in 55 objects
order 23 (8MiB objects)
block_name_prefix: rbd_data.14a96265b3e8c6
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
flags:
create_timestamp: Mon Nov 25 08:30:18 2019
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd –pool images status 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
Watchers: none
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd –pool images status 8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap
rbd: snapname specified for a command that doesn’t use it
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd children images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap
volumes/volume-cd2aa395-f9fc-4b26-aa7e-a34d54cd5645
Volume attached to image snap was removed properly ! Now delete the same …
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd rm volumes/volume-cd2aa395-f9fc-4b26-aa7e-a34d54cd5645
Removing image: 100% complete…done.
Verify …
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd children images/8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd –pool images status 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
Watchers: none
(overcloud) [heat-admin@overcloud-controller-0 ~]$ rbd –pool images status 8b0f9c33-44e7-44eb-8a06-87999dbdeffd@snap
rbd: snapname specified for a command that doesn’t use it
Now remove the image from Openstack Glance registery…
(overcloud) [heat-admin@overcloud-controller-0 ~]$ glance image-list
+————————————–+————–+
| ID | Name |
+————————————–+————–+
| f3e582b9-42a2-44a1-96fa-ca9b5fa3697e | cirros-image |
| e7b16aa1-11c7-42ec-81f6-37c49b64e108 | snap-test001 |
| 8b0f9c33-44e7-44eb-8a06-87999dbdeffd | Ubuntu |
+————————————–+————–+
(overcloud) [heat-admin@overcloud-controller-0 ~]$ glance image-delete 8b0f9c33-44e7-44eb-8a06-87999dbdeffd
(overcloud) [heat-admin@overcloud-controller-0 ~]$ glance image-list
+————————————–+————–+
| ID | Name |
+————————————–+————–+
| f3e582b9-42a2-44a1-96fa-ca9b5fa3697e | cirros-image |
| e7b16aa1-11c7-42ec-81f6-37c49b64e108 | snap-test001 |
+————————————–+————–+
All that assumes I have admin on the cluster. I cannot log in to a controller, I am just a customer of the OpenStack deployment.