Another snapshot post. Who new snapshots could be this awesome.

Again, the situation is VCB failing to remove the snapshot it took of a VM when the backup job that kicked it off failed (because of lack of disk space in this case).

First, vcbSnapshot was used to try and remove it via the VCB proxy manually. For one reason or another, this failed and a Consolidate Helper- 0 snapshot is left behind. In the VI Client, it will show no snapshots in Snapshot Manager but a check of the VMDK targets show that the VM is indeed pointing to delta disks.

Tried taking a new snapshot manually to confirm the .vmsd file is not corrupt. The chain appeared good. Used:
vmare-cmd .vmx removesnapshots
and received the error:
unable to access file since it is locked

While looking into the problem, found these excellent resources for the undocumented vmkfstools -D switch which dumps file info to /var/log/vmkernel. It can be used to identify the owner of a file lock which could be causing the above error when trying to commit back the snapshot delta files:

http://vi3.org/download/guides/english/vi3/StoppingaVMandReleasingFiles.pdf
http://blog.core-it.com.au/?p=477

In this case, however, I didn’t go through the process – instead took the coward’s way out and used: service mgmt-vmware restart on the host that the VM was running on and then used vmware-cmd to commit the snapshots – worked fine this time.

So, I would expect that if I had used the -D switch on the VMDK’s attached to the VM, the owner would have eventually been the ESX Host Agent which would be locking the file on behalf of the VCB request. Must remember to go through the process next time to find out for sure…