About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.
A failed verify means that at least one chunk a snapshot needs could not be read, was missing, or did not match its recorded size or checksum. In the project's source the snapshot is then marked as failed and a corrupt chunk file is renamed to <digest>.<n>.bad, with n from 0 to 9. Do not delete anything yet. First find which chunk and snapshots are affected and why the storage returned bad data, then repair from a good copy or take a new backup.
What the error means
Proxmox Backup Server stores data as chunks. A chunk is possibly encrypted data with a CRC-32 checksum at the end, identified by the SHA-256 checksum of its content. Each snapshot has a manifest (index.json) listing its files with sizes and checksums, and index files (.fidx or .didx) that give "an ordered list of references to the chunks that the original file was split into". Chunks are shared between snapshots, so one bad chunk can fail many snapshots.
A verify job reads every chunk a snapshot references and compares it with what the index expects. The documentation states that verification ensures backup data is intact, can be scheduled as verify jobs, and can be run on a whole datastore, a group or a snapshot.
In the public source, the task log lines you may see include:
can't verify chunk, load failed - <reason>: the chunk file could not be read.can't verify missing chunk with digest <digest>: used for S3-backed datastores when the object is not there.corrupt chunk renamed to <path>: the bad chunk has been moved aside.chunk <digest> was marked as corrupt: a later snapshot in the same run references a chunk already found bad.verify <datastore>:<snapshot>/<file> failed: <reason>andchunks could not be verified.- A summary line ending in
(<n> errors)per file.
These strings come from the source, not the user guide. They read the same on the main branch and in tag v4.2.10, the newest tag. The documentation describes 4.2.11-1, which has no tag, so releases can still differ.
Two limits matter. For encrypted chunks the server holds only the checksum of the plaintext, so "only the CRC-32 checksum gets checked" without the key. And a renamed .bad chunk is tracked separately by garbage collection, whose status counts bad chunks removed and bad chunks still present.
Likely causes
| Cause | What you see | How to confirm |
|---|---|---|
| Storage returned wrong data (bit rot, failing disk, controller fault) | (<n> errors) on a few chunks; load or checksum failures; the same datastore fails again after repair |
Storage health tools for your disks or pool; the documentation recommends re-verifying monthly because drives degrade over time |
| Chunk files deleted or lost (manual clean-up, partial copy, unmounted or restored storage) | can't verify chunk, load failed with the reason the load returned; a missing file is a likely reason, so expect many chunks at once (this link is reasoning, not a documented rule) |
List the chunk path <datastore>/.chunks/<first 4 hex digits>/<digest>; check whether the datastore was recently copied, moved or restored |
| Interrupted write (power loss or hard shutoff) | The backups written around the event fail verify | Compare the failure time with host events; check the datastore sync-level setting |
| Ransomware or tampering on the datastore | Many snapshots fail at once, including old ones (this pattern is reasoning, not a documented rule) | The documentation says encrypted backups no longer match their recorded SHA-256 checksums, so verification fails |
| Index and chunk encryption mismatch | chunk CryptMode ... does not match index CryptMode ... |
Inspect the chunk with proxmox-backup-debug (below) |
The documentation's tuning section says the sync-level default, filesystem, triggers a syncfs after a backup but before the task returns OK, while file runs an fsync on every chunk insertion. It also notes that the underlying device must protect itself against power loss.
Prove the cause before changing anything
Start with checks that only read.
- In the web interface, open the datastore's Content tab and look at which snapshots show a failed verification, then open the verify task log. Note the first error, not the last, and write down the chunk digests.
- Look at the snapshot's referenced chunks:
proxmox-backup-debug inspect file drive-scsi0.img.fidxlists the chunks a.fidxindex references. A good result is a list of digests. A bad result is an error reading the index. - Find which index files use a given chunk:
proxmox-backup-debug inspect chunk <digest> --reference-filter /path/in/a/datastore/directory. Without a filter it prints only the CRC and encryption status. - Check the file exists and its size looks right, using normal file tools on
<datastore>/.chunks/<4 hex digits>/<digest>. - Check the storage layer with your storage tools. Proxmox's administration guide shows
zpool status -vfor watching a ZFS resilver.
Risk: running the verify command is not read-only. proxmox-backup-manager verify <datastore> --read-threads 1 --verify-threads 4 --ignore-verified false writes the verify result into each snapshot's manifest and, per the source, renames chunks it finds corrupt. Run it deliberately, not as a harmless look. Renaming moves the file aside; it does not delete it.
Fix
If you have a good copy on another Proxmox Backup Server. The documentation describes a sync job option, resync-corrupt, which "will re-sync all snapshots that have failed to verify during the last Verification". Run a verification first, then the sync job. It "needs to check the manifests of all snapshots in a datastore and might take much longer than regular sync jobs". The protected flag of snapshots is not synced, so set it again on the target if you need it.
Risk: the re-sync rewrites the snapshots that failed verification on the target with the copy from the other server. If that copy is also damaged, the target gains nothing, so verify the other server first.
If the source still holds the data. Take a fresh backup. In the source, the server does not use a failed latest snapshot as the starting point for the next one, so the new backup should not depend on the damaged one. The documentation does not state how much data the client re-sends in that case. Watch the transfer and the duration.
If nothing intact remains and you need the readable parts. Try proxmox-backup-debug recover index drive-scsi0.img.fidx /path/to/.chunks. The --skip-crc option "will speed up the process slightly and allow for trying to restore (partially) corrupt chunks". The documentation recommends always trying without --skip-crc first. Restore to a new location, not over live data.
Removing the failed snapshot. Disruptive step: proxmox-backup-client snapshot forget <snapshot> deletes the snapshot. There is no way back from this command. The documentation says "The prune command removes only the backup index files, not the data", so space is freed when garbage collection runs. Protected snapshots are skipped, because the protected flag prevents pruning or manual deletion. Do this only once the affected restore points are replaced.
Garbage collection. Disruptive step: proxmox-backup-client garbage-collect deletes chunks that no snapshot references and that are older than a cutoff of 24 hours and 5 minutes, or the oldest running backup. No way back. Wait until the repair is finished and verified.
If the cause is hardware, fix or replace it before repairing. The same bad storage will return bad data again.
Confirm the fix
- Re-verify the affected snapshots, or the datastore with
--ignore-verified falseso already verified snapshots are checked again. Expect(0 errors)for each file and no new.badfiles. - Restore a recent snapshot to a new guest, not over the live one. The documentation recommends testing restores this way.
- Check that the next scheduled backup finishes and that its snapshot verifies.
Example (illustrative): backup01 holds vm/100, a 100 GiB disk stored in 4 MiB chunks. 100 × 1,024 ÷ 4 = 25,600 chunks. Verification finds 3 bad chunks. Those 3 chunks are 12 MiB, 0.012% of the 25,600 chunks. If they sit in an unchanged part of the disk, every snapshot of vm/100 since they were first stored references them, so all snapshots fail. A re-sync from the remote copy restores the good chunks. A single verify pass reads each unique chunk once per run, because chunks already verified in the run are skipped, so the first pass costs about one read of the unique data: 100 GiB at 200 MiB/s is 102,400 ÷ 200 = 512 seconds, a little under 9 minutes.
What to watch afterwards
- Run a verify job that checks new backups hourly or daily, and a second job that re-verifies everything weekly or monthly. The documentation recommends re-verifying at least monthly, even when earlier verification passed.
- Turn on notifications for verification and garbage collection results.
- Compare error counts over time. Errors that return after repair point to hardware.
- Keep a copy of the datastore on another server. The documentation's 3-2-1 guidance recommends off-site copies and describes sync jobs that do not remove snapshots that vanished from the source.
Read next
Related articles you may want next.
- Backup & RecoveryWhat restic and Borg repository checks prove, and what only a restore canWhat restic check, --read-data, borg check and --verify-data actually prove, what they cost, and the restore test that covers what they cannot.Read article
- Backup & RecoveryFinding a bad backup before the day you need itWhat hashes, signatures and sample restores detect in backups, which faults they miss, and when to treat a copy as unusable.Read article
- Backup & RecoveryA successful backup job is not a successful restoreWhy a finished backup job does not prove you can restore, and which checks show the copy can actually be opened and used.Read article
Risks and limits
The documentation does not list the log strings above. They come from the project's public source (main branch and tag v4.2.10) and may change. Verification of encrypted chunks without the key checks only the CRC-32 checksum. Verification tells you the stored data matches its checksums, not that the guest will boot after a restore. Do the restore test.
These commands have not been run or lab-tested.
Sources
- Maintenance Tasks (pruning, garbage collection, verification) — Proxmox Server Solutions, footer 2019-2025, accessed 17 November 2026
- Technical Overview — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Backup Client Usage — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Managing Remotes and Sync — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Storage (datastore tuning and ransomware protection) — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Host System Administration — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Source file src/backup/verify.rs — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Source file pbs-datastore/src/chunk_store.rs — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Source file src/api2/backup/mod.rs — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Source file src/backup/verify.rs, tag v4.2.10 — Proxmox Server Solutions, date not shown, accessed 17 November 2026
- Source file pbs-datastore/src/chunk_store.rs, tag v4.2.10 — Proxmox Server Solutions, date not shown, accessed 17 November 2026
This article was originally published on .