Australian cloud for business software — built and run in Sydney since 2002

NetLeverage
NetLeverage

restic forget kept the wrong snapshots: how --keep-daily, --keep-weekly and --keep-last combine

Technical article
Backup & Recovery

About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.

When restic forget keeps a different set of snapshots from the one you expected, check three documented rules first. The --keep-* options are combined with OR, not added together. Calendar options count only periods that contain at least one snapshot. The policy runs separately for each group of snapshots, and by default a group is one host plus one set of backup paths.

Test any policy with --dry-run before you let it remove anything. The dry run prints which snapshots would be kept, and why.

What the error means

There is no error message here. The symptom is a result: fewer restore points than you wanted, more than you wanted, or a particular day missing. Restic's documentation sets out the rules that produce it.

All options are ORed. The restic documentation on removing snapshots says: "The processed snapshots are evaluated against all --keep-* options but a snapshot only needs to match a single option to be kept (the results are ORed)." A policy of --keep-last 3 --keep-daily 7 --keep-weekly 4 does not keep 3 + 7 + 4 snapshots. One snapshot can satisfy several options at once, so the total is smaller.

Calendar options count periods that have snapshots. --keep-daily n means: "for the last n days which have one or more snapshots, keep only the most recent one for each day." The same wording applies to hours, weeks, months and years. A day with no backup is not counted, so the window stretches back to find n days that do exist.

Calendar boundaries are natural, not relative. The documentation says these options "work on natural time boundaries and not relative to when you run forget. Weeks are Monday 00:00 to Sunday 23:59, days 00:00 to 23:59". It does not say which time zone defines midnight. In the v0.19.1 source, each snapshot is placed in a day or week using the calendar fields of its own recorded timestamp, with ISO week numbering. Snapshots taken from hosts in different time zones can therefore land on different days from the ones you expect.

The newest snapshot in each period wins. Restic keeps "the most recent one for each" period, not the first. A weekly snapshot is the newest one in that Monday-to-Sunday week, which is not necessarily taken on a Sunday.

The oldest snapshot can be kept as a bonus. The manual page for forget says that if there are not enough snapshots to fill a duration-related option, "the oldest snapshot in the group is kept additionally".

Policy runs per group. The documentation says restic "groups them by their host name and paths" and applies the policy to each group individually, "a safety feature to prevent accidental removal of unrelated backup sets". The default is --group-by host,paths.

--keep-within is relative to the latest snapshot, not to today. The manual page describes it as "relative to the latest snapshot". Snapshots with a timestamp in the future are ignored by the duration options and not removed.

Likely causes

Cause What you see How to confirm
Counts expected to add up Fewer snapshots kept than the sum of the numbers --dry-run output lists a reason for each kept snapshot; several reasons share one snapshot
A missed backup day --keep-daily 7 reaches back more than 7 calendar days Compare the oldest kept daily snapshot with today's date
Weeks run Monday to Sunday The weekly snapshot is the last one of that week, not a fixed weekday Check the weekday of each kept weekly snapshot
Backup paths or host name changed Old snapshots never get removed, or the new set seems to have no history restic snapshots lists two different Host or Paths values for what you think is one set
--keep-within read as "within the last N days" Nothing is removed after a long gap in backups Duration counts back from the newest snapshot, not from now
Tag policy or --tag filter Snapshots with other tags untouched, or tagged ones kept for ever The policy lists --keep-tag or --tag; check tags in restic snapshots
forget without prune Snapshots gone from the list but the repository did not shrink Space is only released by prune (or forget --prune)

Prove the cause before changing anything

These commands come from the restic documentation and manual pages. The -v flag makes forget print a heading for each group.

restic snapshots
restic forget -v --keep-last 3 --keep-daily 7 --keep-weekly 4 --dry-run --no-lock

restic snapshots lists every snapshot with its time, host, tags and paths. --dry-run ("do not delete anything, just print what would be done") prints the policy, the snapshots to keep with a Reasons column, and the snapshots to remove. In the v0.19.1 source, forget takes an exclusive lock even for a dry run, so a dry run can fail with "repository is already locked" while a backup runs. --no-lock is accepted only together with --dry-run and skips the lock. Without the lock, nothing stops a backup or prune from changing the repository while the plan prints, so treat the listing as a snapshot of that moment. A lock error needs its own diagnosis before you unlock anything.

Read the output in this order:

  1. Count the groups. In restic 0.19.1, forget prints a group heading only with -v (verbose), in the form snapshots for (host [fs01], paths [/srv/shares]):. Without -v, read the Host and Paths columns of the keep and remove tables instead. A group you did not expect points to a changed host name or path.
  2. For each kept snapshot, read the reason. "last snapshot", "daily snapshot" and "weekly snapshot" on one line mean one snapshot is doing several jobs.
  3. Find the oldest kept snapshot in each group. If it is older than the policy seems to allow, look for a gap in the backup dates.

Example (illustrative): fs01 backs up /srv/shares to a repository on backup01 every day at 01:00. The backup for Wednesday 11 November 2026 was missed. On Tuesday 17 November the policy is --keep-last 3 --keep-daily 7 --keep-weekly 4. The repository holds 29 snapshots, from 19 October to 17 November.

Kept snapshot Why
Tue 17 Nov last, daily, weekly (newest in week 47)
Mon 16 Nov last, daily
Sun 15 Nov last, daily, weekly (newest in week 46)
Sat 14, Fri 13, Thu 12 Nov daily
Tue 10 Nov daily (the missed Wednesday is skipped, so the seventh day with a snapshot is the 10th)
Sun 8 Nov weekly (newest in week 45)
Sun 1 Nov weekly (newest in week 44)

That is 9 snapshots kept and 20 removed, not 3 + 7 + 4 = 14. The 7 daily snapshots span 8 calendar days because of the gap. This table follows the rules in the restic documentation and the v0.19.1 source.

Fix

Change the policy, then test it again with --dry-run. Do not run the real command until the dry run lists the snapshots you want.

If the counts were expected to add up: raise the numbers that matter. To keep more than one snapshot per day, use --keep-last or --keep-hourly, because a daily option keeps one per day.

If a gap is stretching the window: accept the documented behaviour, or use --keep-within-daily. The documentation's own example shows that after a missed day, --keep-daily 5 kept a snapshot from the previous Friday, while --keep-within-daily 7d kept only daily snapshots from the last seven days of the newest snapshot. Neither is wrong; pick the one that matches the promise you make about restore points.

If groups are split: decide whether the old path or host should keep its own history. Restic's --group-by option accepts host, paths and tags. The documentation says "one would normally set the --group-by option for the backup command to the same value" and that --group-by '' disables grouping. Risk: with grouping disabled, the policy applies to every snapshot in the repository regardless of host, so a policy meant for one server can remove another server's history. Back out by setting --group-by host,paths again, but snapshots already removed cannot be restored by restic.

If tags are involved: check the policy against the documented rules. Several --tag arguments are ORed; tags separated by commas inside one argument must all be present. --keep-tag '' matches untagged snapshots only.

To apply the policy:

restic forget --keep-last 3 --keep-daily 7 --keep-weekly 4 --prune

Disruptive step: forget removes snapshot records, and --prune ("automatically run the 'prune' command if snapshots have been removed") then deletes data that no remaining snapshot uses. There is no way back inside restic. The only protection is a separate copy of the repository taken beforehand. During a prune, the restic documentation says "the repository is locked and backups cannot be completed", so schedule it away from backup windows.

Confirm the fix

  1. Run the dry run again and compare the kept list with what you intended.
  2. After the real run, restic snapshots should show the same set that the dry run listed.
  3. The documentation advises running restic check after pruning "to make sure you are alerted, should the internal data structures of the repository be damaged". Then restore a sample from the oldest kept snapshot into a scratch directory and compare it with the source. A policy that keeps the right snapshots does not prove they restore.

What to watch afterwards

  • Keep the policy in one place, such as a script under version control, so a changed number is a visible edit.
  • Record how many snapshots each group should hold at its smallest and largest. A count outside that range means a group split, a gap or a policy change.
  • Run the dry run after any change to a backup path, host name or tag, before the next scheduled forget.
  • Be careful with repositories that accept only appended data. The documentation warns that someone who can add snapshots with recent timestamps can make a calendar policy discard the genuine ones, and recommends --keep-within for such repositories.

Risks and limits

  • These commands have not been run or lab-tested. They come from the restic documentation, manual pages and the v0.19.1 source.
  • The documentation pages show no date. Policy behaviour comes from the restic 0.19.1 source. Earlier or later versions may differ, for example --unsafe-allow-remove-all exists only from restic 0.17.0.
  • The restic documentation does not say which time zone defines midnight for the daily and weekly buckets. The statement about each snapshot's own timestamp comes from the restic 0.19.1 source.
  • --keep-last 0 does not remove everything. The documentation says restic "refuses to act on an 'empty' policy" and removes nothing.
  • A hand-worked example is not a substitute for your own dry run on your own snapshot list.

Sources