Nutanix: Unable to delete container / VDisk(s) not marked for removal

If you are unable to delete a container within Prism this is mostly related to remaining files on the datastore / container. When you try to delete the container you will get an error message like “Container {container-name} contains VDisk(s) 123, 456, 789 not marked for removal.” 1_prism_error

Mostly it should help if you remount the datastore to your vSphere hosts and browse the datastore and manually delete all remaining files.
Magnus from Nutanix has already a step by step guide on his blog, which you can find here.

In rare cases this may doesn´t help either and you still getting the same error even if your datastore looks empty to you (from the hypervisor perspective). In my case it have been some snapshot zombies, which remained from an metro availability setup.

Following you find a step by step guide how to get rid of the remaining vdisks via ncli. Please use it carefully and on your own risk. In a productive environment I would recommend you to let you help from their awesome support.

1.) Connect to a CVM of your Nutanix cluster via SSH to get the ID of the affected container with ‘ncli container list ‘. You need the number after the two colons, like you can see on the screenshot.
2_container_list

2.)
Put this ID on the end of the command ‘vdisk_config_printer –container_id= ‘ to get a list of all vdisks which are existing on the related container. On the screenshot you notice the vdisk_ids are matching to the numbers shown in the error message I posted on the beginning.
All the lines blow vdisk_id until the next vdisk_id line are releated to a single vdisk with different configuration options / infos etc. In this options one important parameter is missing which allows the container delete command to be successful.
I coudn´t get all vdisks on a single screenshot so I listed it again (second screenshot) but did a grep in addition to just show you the line before “creation_time_usecs”. For the first 3 vdisks (which are the one which can´t be deleted) it´s just an “}”, while all others have the parameter “to_remove: true”.
3_vdisk_list
4_to_remove_missing
3.) You can modifiy this lines with the command ‘edit_vdisk_config –vdisk_id=12345 –editor=vim ’. Within vim you can now edit all lines. You need to insert “to_remove: true” on the place of the cursor on the screenshot (I don´t know if the order matters, just keep it like default). Remember to save the file with ‘:wq ’ when you exit vim.
5_vim
4.) Repeat this steps for each vdisk which is listed in your error message.

5.) Now you should be able to delete the container successfully. If you are already on the shell you can also do this via ‘ncli container remove id=12345 ignore-small-files=true ’ (The id here needs to be the complete one, check the screenshot for an example).
6_example

Comments

  1. Thanks for the article. Helped me.
    Please correct command in the first step “ncli container list” instead of cluster list.

  2. Hello,

    Just noticed that your screenshot show ncli container list while the text says ncli cluster list.
    Your text is wrong – the screenshot is right.

  3. vMario156 says:

    Thanks for the hint on the typo 🙂
    I updated the command in the text.

    Cheers Mario

Speak Your Mind

*