A snapshot removal or consolidation task is stuck, or fails with a lock error

In vCenter, a "Remove snapshot" or "Consolidate" task sits at one percentage for a long time, or it ends with an error such as "Failed to lock the file", "Unable to access file since it is locked" or "Unable to consolidate virtual machine snapshots due to file lock". The VM Overview may then show that virtual disk consolidation is required. Sometimes a snapshot disk stays in the VM folder.

Two situations produce that picture. In the first the task is working and is slow. In the second another process holds a lock on a disk file and the task cannot finish. The first needs patience; the second needs you to find and release the holder.

This article shows how to tell them apart, find the holder, clear it through scoped recovery decisions on VMFS, NFS and vSAN, and why you must not cancel the task. Commands and numbers come from the Broadcom pages under Sources; our own reasoning is labelled. We have not run these procedures in your environment: treat every procedure here as an untested procedure, run in a lab first.

What is actually happening

A snapshot freezes a virtual disk and sends new writes to a separate delta file. Removing the snapshot writes the delta's blocks back into the parent disk. Broadcom describes this as very I/O intensive, with duration set by total delta size, chain depth, the storage array and host load (KB 338534).

A lock is how a host claims a file so nobody else changes it. A running VM holds exclusive locks on its own files, and the base disk of a VM with snapshots carries a read-only lock (KB 314365). Consolidation succeeds only when the VM's own vmx process is the sole holder (KB 374141).

The usual second holder is a backup proxy, a machine that reads a VM's disks during a backup. When it attaches a snapshot disk to itself (the hot-add method) the host locks that file for the proxy, and if the backup product does not release the disk afterwards, the lock stays (KB 321365).

A long task is a separate matter. ESXi sends new writes to a helper delta, then commits it to the base disk while the VM is briefly frozen (stunned). If the commit would take over 15 seconds it waits and retries, up to 10 iterations. KB 323397 describes a changed behaviour that gives up after the ninth iteration; the page does not say which versions have it. While removal runs, power operations and vMotion on that VM cannot run (KB 323397).

How to prove it

All steps here only read.

1. Confirm in the backup console that no backup job is running against the VM. KB 321365, KB 374141 and KB 313833 each start with this check.

2. Decide whether the task is working. Note the task and percentage in vCenter under the VM, Monitor, Tasks and Events. On the host, list the delta files, then watch the datastore device in esxtop (press u, read MBREAD/s and MBWRTN/s) (KB 338534):

ls -lh /vmfs/volumes/DATASTORE_NAME/VM_NAME | grep -E "delta|sparse"
esxtop

The pages give no universal time limit for a stalled task. Correlate the target VM's logs, file activity and task state; steady shared-device writes may belong to another VM, and an idle sample does not prove a lock. These are diagnostic clues, not a definitive test. The cited esxtop method does not apply to NFS (KB 338534).

3. Read the VM's vmware.log. KB 323397 shows an estimate line, SnapshotVMXNeedConsolidateIteration: Size of helper disk ... approx. time required for consolidating helper disk = ... sec. A lock failure appears as Foundry operation failed with system error: Device or resource busy (16), and hostd.log shows Failed to delete disk ... Device or resource busy (KB 321365).

4. Find the lock owner. On vSphere 8.0 U2 and later the owner shows in the status of a failed power-on task, under VM, Monitor, Tasks and Events, Tasks (KB 314365, KB 316576); the pages do not say a failed consolidation task shows it. For a consolidation, or on earlier versions, use the host. On VMFS, from the VM's folder (this loop fails on file names containing spaces):

cd /vmfs/volumes/DATASTORE_NAME/VM_NAME
ls | while read x; do vmfsfilelockinfo -p $x | grep -i "is locked"; done

A powered-on VM's files should be locked by the host it runs on; a powered-off VM should produce no output, so any output is a stale lock; two MAC addresses on one file means an abnormal second holder (KB 313833). To resolve the MAC to a host in one command, add -v VCENTER_IP -u USER to vmfsfilelockinfo -p FILE (KB 314365). Otherwise match it under each host's Configure, Physical adapters, or with esxcli network ip neighbor list (KB 313833). Lock modes: 0 none, 1 exclusive, 2 read-only (base disk of a running VM with snapshots), 3 multi-writer (KB 314365).

5. On the locking host, name the process and the VM:

lsof | egrep 'Cartel|VM_NAME-000001-delta.vmdk'
esxcli vm process list

The Cartel ID in the lsof line matches the VMX Cartel ID in the esxcli list, whose entry names the VM (KB 314365). Look for a vmx process that is not the impacted VM's own (KB 374141). A holder named hostd or vpxa-worker is a management agent (KB 313833).

6. Check the VM for a stuck task (KB 316576):

vim-cmd vmsvc/getallvms | grep -i VM_NAME
vim-cmd vmsvc/get.tasklist VMID
vim-cmd vimsvc/task_info TASK_ID

7. On NFS, vmfsfilelockinfo and vmkfstools -D show no external locks (KB 412870). The lock files are hidden, with names starting .lck-, so list the directory with dotfiles and filter. This is plain shell; we did not test it on an ESXi shell. KB 412870 prints ls -larth *.lck*, which a shell glob may not expand for hidden names.

ls -lah | grep lck
hexdump -C .lck-XXXX

The hexdump output shows the owner's hostname (KB 412870, KB 316576). On a powered-off VM with no other access, no .lck file should exist (KB 316576).

The causes, in order of how direct the evidence is

No official page ranks causes by frequency.

A backup proxy still has the disk attached (hot-add). Step 5 shows a foreign vmx, and the proxy's settings list a disk belonging to the failing VM (KB 321365, KB 374141).

A backup process holds the disk over a network or storage transport. With NBD, NBDSSL or SAN transport a hung process on the proxy keeps it open; there is no extra disk on the proxy (KB 321365).

A stuck management agent. The holder is hostd or vpxa-worker, or step 6 shows a task that never ends (KB 313833, KB 316576). On NFS, stale .lck files stay when hostd or vpxa go stale (KB 412870).

A chain that has grown to its limit. vmware.log shows msg.disklib.numLinks.maxReached when a branch has 255 or more redo logs, which follows a snapshot backup that keeps failing to release its lock (KB 374141).

Recovery decisions and escalation

Complete the diagnostic checks before choosing a branch. These are decision boundaries, not tested repair commands. Before any modifying procedure, identify the exact target, storage type, host, holder and active jobs; preserve evidence and a demonstrated restore route; assess capacity, outage and other workloads. Stop if ownership or task progress is uncertain.

• Progressing merge: correlate the target's logs and file activity with its task. Shared-device writes may come from other VMs. Observe an active merge rather than cancelling it, restarting agents or forcing a power operation.
• Confirmed backup-proxy holder: ask the backup supplier to release the exact attachment using its supported workflow, without deleting datastore files. Normal removal and the ESXi 7.x/8.x removal-refused case in KB 375284 require different paths. The latter documents a powered-off proxy, consolidation and subsequent detachment; use it only after specialist confirmation of the matching error and prerequisites. See https://knowledge.broadcom.com/external/article/375284.
• vSAN lock remains: KB 321365 contains a vSAN-specific lock-file operation. It does not make wildcard movement appropriate for an unidentified namespace or supply every environment-specific prerequisite. Preserve the exact namespace, file identity and owner evidence for Broadcom; lock-file modification is withheld here pending a case-specific expert procedure.
• Management-agent holder: a separate host change plan must cover running operations, all affected VMs, management dependencies and recovery access. Follow the build-specific restrictions in KB 320280, rather than restarting both agents by default. If HA monitoring is temporarily changed, restore its recorded previous state and verify protection after recovery.
• NFS lock or unexplained VMFS lock: retain the storage-specific diagnostic evidence and escalate. Do not import the vSAN branch into NFS or VMFS. A host shown as disconnected may still be running. Lock movement, scratch-file deletion, maintenance entry, reboot and forced VM termination require their own validated procedures; none is the next automatic step.

After an approved branch, verify the original fault has cleared, the intended disk chain is intact, applications are correct and backup protection works. Stop after success. A recurring lock requires investigation of backup cleanup, not repeated consolidation.

What not to do

• Cancel the task because it looks stuck. Broadcom warns that cancelling a running deletion or consolidation can lead to data corruption (KB 318825), and that once started it cannot be stopped (KB 338534). A stalled percentage is not evidence of a stall.
• Tick "Delete files from the datastore" when removing the disk from the proxy. That removes the disk files from the datastore (KB 321365).
• Delete delta files by hand. The snapshot files alone are not enough to restore a VM once the base disks are gone (KB 318825). Even a documented scratch-file case needs exact path and object confirmation by a specialist; this article provides no deletion procedure.
• Do not move or delete lock files from this general guide. VMFS, NFS and vSAN require different ownership and applicability checks; a vendor-specific exception is not a generic release instruction.
• Restart agents with services.sh restart or from the DCUI on hosts running vSAN, LACP, NSX or shared graphics. KB 320280 says to restart single services only.
• Restart agents without disabling HA Host Monitoring first (KB 316576).

Acceptance checklist

These are required checks, not completed test results.

• [ ] Target logs/file activity correlated with task; shared I/O not treated as proof.
• [ ] Exact holder and storage-specific branch established.
• [ ] Backup, capacity, outage and other workloads assessed before changes.
• [ ] Uncertain ownership or lock modification escalated.
• [ ] Host changes have independent access and HA restoration checks.
• [ ] Success verified once, then stopped; recurring fault investigated.

Sources

1. KB 321365, Snapshot consolidation fails due to locks held by third-party backup software - VMware vSphere. https://knowledge.broadcom.com/external/article/321365. Scope: ESXi 7.x to 9.x, vCenter 7.x to 9.x.
2. KB 374141, Snapshot consolidation failure: Failed to lock the file error. https://knowledge.broadcom.com/external/article/374141. Scope: ESXi and vCenter 8.x and 9.x.
3. KB 412870, VM Backup jobs and disks consolidation tasks fail due to stale NFS locks - VMware vSphere. https://knowledge.broadcom.com/external/article/412870. Scope: ESX 8.x, also lists 9.x.
4. KB 316576 (legacy ID 2107795), Troubleshooting issues resulting from locked virtual disks. https://knowledge.broadcom.com/external/article/316576. Scope: ESXi 6.x, 7.x, 8.x.
5. KB 323397, Snapshot removal stops a virtual machine for a long time. https://knowledge.broadcom.com/external/article/323397. Scope: ESXi 8.0, 7.0, 6.7, 6.5, 6.0.
6. KB 338534, Estimate the time required to consolidate virtual machine snapshots. https://knowledge.broadcom.com/external/article/338534. Scope: ESXi 7.x and 8.x.
7. KB 318825, Best practices for using VMware snapshots in the vSphere environment. https://knowledge.broadcom.com/external/article/318825. Scope: ESXi 8.x, ESX 9.x, vCenter 8.x and 9.x.
8. KB 314365, Investigating Virtual Machine file locks on ESXi Host(s). https://knowledge.broadcom.com/external/article/314365. Scope: ESXi 7.x, 8.x, 9.x.
9. KB 313833, VMware virtual machine file lock on VMFS datastore. https://knowledge.broadcom.com/external/article/313833. Scope: vSphere 6.x, 7.x, 8.x.
10. KB 320280, Restarting Management Agents in ESXi. https://knowledge.broadcom.com/external/article/320280. Scope: ESXi 7.x, 8.x, 9.x.

Additional source checked 24 October 2026: https://knowledge.broadcom.com/external/article/375284