Australian cloud for business software — built and run in Sydney since 2002

NetLeverage
NetLeverage

PostgreSQL will not start: 'could not locate a valid checkpoint record'

Technical article
Backup & Recovery

About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.

PostgreSQL has read its control file, found the position of the last checkpoint it recorded, and cannot read a valid checkpoint record from the write-ahead log (WAL) at that position. It stops with a PANIC rather than guess. Common causes include WAL files that were deleted, left out of a copy or damaged by storage, or a file-level copy of a running cluster that was restored without the files that make it consistent. Copy the whole data directory before you do anything else, then look for a restore path. pg_resetwal can get the server to start, but its own documentation calls it a last resort and tells you to dump, run initdb and restore immediately afterwards.

What the error means

When PostgreSQL starts after an unclean stop, recovery begins by reading the file pg_control, then the checkpoint record it points to, then replaying WAL forward from the location that record names. WAL lives in the pg_wal directory as segment files, normally 16 MB each. If the checkpoint record cannot be read, there is no safe starting point for replay.

In the REL_18_STABLE source, InitWalRecovery() in xlogrecovery.c has two branches that matter here:

  • No backup_label file present: the checkpoint named in pg_control is read. If that fails, the server raises a PANIC with the message could not locate a valid checkpoint record at X/X, where X/X is a WAL location.
  • backup_label present: the checkpoint named in that file is read instead. If that fails, the server raises a FATAL with could not locate required checkpoint record at X/X. Its hint starts with the server's own restore advice: "If you are restoring from a backup, touch "%s/recovery.signal" or "%s/standby.signal" and add required recovery options." It then says that if you are not restoring from a backup you can try removing backup_label, and warns that removing it "will result in a corrupt cluster if restoring from a backup".

The two messages are different faults. The PANIC means the checkpoint that pg_control points to cannot be read, as after a crash or a copy taken while the server was running. The FATAL means the checkpoint named in backup_label cannot be read; the cause can be a restore from a backup whose WAL is incomplete, or a restore that is missing recovery.signal and its recovery options. Match your log line to the right branch before choosing a fix.

The wording above is from the 18 source. Earlier releases may word the message differently; that was not checked.

Likely causes

Cause What you see How to confirm
WAL files deleted or emptied from pg_wal (for example to free disk space) Server stopped or crashed first; pg_wal is nearly empty or missing recent segments ls of pg_wal shows few or no files; the server log shows the PANIC at the next start
File-level copy of a running cluster, restored without the WAL or without backup_label New copy from a copy tool or VM clone will not start The documentation says a file system copy is only usable if the server was shut down, or if it is a consistent snapshot that includes the WAL; check how the copy was made
Restore that cleared pg_wal but never supplied archived WAL Restore completes, startup fails The recovery procedure removes the old pg_wal contents and expects restore_command and recovery.signal to supply WAL; confirm both exist
Storage fault or lost files Other I/O errors in the operating system log; unreadable segment files OS logs, SMART or array health, and a read test of the segment files
Mismatched files from different points in time (for example pg_control from one copy and WAL from another) Copy assembled by hand from several sources Compare the origin of each directory; check the dates of pg_control and the WAL files

PostgreSQL does not document the last row as a cause; it follows from the startup sequence described above.

Prove the cause before changing anything

Everything below is read-only. These commands are taken from the PostgreSQL documentation and have not been run.

  1. Stop and preserve. Do not start the server repeatedly. Make a complete copy of the data directory, including pg_wal and any tablespaces, onto separate storage before anything else. The documentation's own example for a file-level copy, which needs the server shut down, is tar -cf backup.tar /usr/local/pgsql/data. If the storage is failing, image it instead of reading it repeatedly.
  2. Read the log line. Note whether you have the PANIC (no backup_label) or the FATAL (backup_label present), and the X/X location.
  3. Check whether a backup_label file exists in the data directory.
  4. List pg_wal. Segment names run in ever-increasing hexadecimal order starting at 000000010000000000000001. Look for gaps, zero-length files and an unexpectedly small directory.
  5. Print the control file. pg_controldata prints write-ahead logging and checkpoint information and needs read access to the data directory: pg_controldata -D datadir. Compare its checkpoint information with the WAL files you have.
  6. Read the WAL. pg_waldump displays WAL records in readable form. By default it searches pg_wal of PGDATA, and -p path points it at another directory. Its documentation warns it "can give wrong results when the server is running", so use it only on a stopped server or on your copy.
  7. Check the archive and the base backup. Find out whether a base backup and archived WAL exist. That decides whether you can restore instead of repairing.

Example (illustrative): on PG01 a disk-space alert led someone to delete the oldest files in pg_wal. The server was then restarted and logged PANIC: could not locate a valid checkpoint record at 0/5000028. pg_wal held three small segment files and no backup_label existed. That points to missing WAL, and the next question is whether a base backup and an archive exist.

Fix

Choose in this order. Each later option gives up more.

1. Restore from a backup (preferred)

If you have a base backup and archived WAL, restore rather than repair. The documented recovery steps are: stop the server, keep a copy of the damaged directory, restore the base backup, remove any files in pg_wal/ that came from it, copy back any unarchived WAL you saved, set recovery settings in postgresql.conf including restore_command, and create a file recovery.signal in the data directory. The server removes recovery.signal when recovery completes. The documentation notes that restore_command must return a nonzero exit status when a file is missing, and that a signal or shell error such as command not found aborts recovery.

A backup made with pg_basebackup -X (stream or fetch) includes the WAL it needs and starts without the archive. The documentation describes such a standalone hot backup as one that cannot be used for point-in-time recovery.

For choosing a recovery target, see the related article on point-in-time recovery.

2. Supply the missing WAL

If the missing segments exist in an archive or in another copy, recovery may work with a correct restore_command and recovery.signal. Do not copy files into pg_wal of your only copy; work on the preserved copy.

3. pg_resetwal (last resort, data-loss risk)

Risky step: this can leave the database inconsistent. Work only on a copy of the data directory. The pg_resetwal page says it "should be used only as a last resort, when the server will not start due to such corruption". After running it on corrupted WAL, "the database might contain inconsistent data due to partially-committed transactions", and you "should immediately dump your data, run initdb, and restore". It also says not to run data-modifying operations before the dump.

Documented points:

  • The server must not be running. It works only with servers of the same major version, and only as the user who installed the server.
  • The data directory must be given with -D; the environment variable PGDATA is not used.
  • -n (--dry-run) prints the values it would change and exits without modifying anything: pg_resetwal -n -D datadir. Run that first.
  • A server that was not shut down cleanly needs -f (--force). Treat the need for -f as the sign you have left safe territory.
  • If the contents of pg_wal are lost entirely, the -l option may need setting by hand, and the page describes how to choose the value.

Then dump everything, create a fresh cluster with initdb, restore, and check the result for inconsistencies. Keep the original directory untouched as a fallback, not as a database to keep running.

Do not

Do not remove backup_label from a restore that needs it. The server's own hint says doing so gives a corrupt cluster. Do not start the server over and over; each start is a chance to change files you may need.

Confirm the fix

  • The server starts and logs that recovery completed, without the PANIC.
  • Run application-level checks, not just a connection test: row counts on key tables, recent transactions you can identify, and a full logical dump that completes.
  • If you restored from backup, compare the recovered data against what the application expects at the recovery target.
  • After a pg_resetwal repair, the real proof is a dump and restore into a new cluster, followed by a check of constraints and application behaviour. A started server alone proves nothing about data integrity.

What to watch afterwards

  • Alert on pg_wal growth and on disk space on the WAL volume, so nobody deletes WAL by hand. Removing files from pg_wal by hand removes the records crash recovery reads.
  • Monitor WAL archiving. A failing archive fills pg_wal; the documentation warns the server will do a PANIC shutdown if the file system holding pg_wal fills.
  • Take base backups that include WAL, and test restoring them on a separate server on a schedule.
  • If copies are taken by snapshot, confirm the snapshot covers the data and WAL together, since the documentation requires simultaneous snapshots when they sit on separate volumes.
  • Recurrence signal: the same error after a fresh restore means the backup itself lacks WAL.

Risks and limits

  • These commands and procedures are taken from the PostgreSQL documentation and have not been run or lab-tested.
  • Version scope is PostgreSQL 18. Option names and message wording may differ in earlier releases.
  • The causes table is a diagnostic aid; the documentation does not list causes for this specific message.
  • The documentation describes a theoretical weak spot: a corrupt pg_control file, noting that scanning WAL for the latest checkpoint is not implemented. This article does not cover repairing pg_control by hand.
  • pg_resetwal cannot recover transactions that lived only in the lost WAL.
  • If storage hardware is failing, imaging the disk comes before any of these steps.

Sources

  • pg_resetwal — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • 28.6 WAL Internals — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • 25.3 Continuous Archiving and Point-in-Time Recovery — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • 25.2 File System Level Backup — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • pg_controldata — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • pg_waldump — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • pg_basebackup — PostgreSQL Global Development Group, PostgreSQL 18.7 documentation, date not shown, accessed 16 November 2026
  • xlogrecovery.c, REL_18_STABLE — PostgreSQL source repository, source file (copyright to 2025), accessed 16 November 2026