Australian cloud for business software — built and run in Sydney since 2002

NetLeverage
NetLeverage

Incremental vs differential backup: what each one stores and what a restore needs

Technical article
Backup & Recovery

About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.

A full backup copies everything that was selected. An incremental backup copies only what changed since the previous backup of any kind. A differential backup copies only what changed since the last full backup. That single difference decides how large each night's backup is and how many separate pieces a restore must find and read in the right order.

Vendors use these three words slightly differently, so the label alone is not enough. The sections below show what each type stores, what a restore needs, and where each type can quietly lose or resurrect data.

What does each backup type store?

The three types differ in their reference point: the earlier backup that "changed since" is measured against.

Type Reference point Size over a week What a restore of the latest night needs Main weakness
Full None. Copies everything selected Same every time That one full backup Largest to read, move and store
Incremental The previous backup of any type Each night is small and roughly constant The last full plus every incremental since it, in order One missing or unreadable link breaks everything after it
Differential The last full backup Grows each night as more has changed since the full The last full plus the latest differential Gets close to full size if the full is old

Bareos documents all three levels in its Director configuration. A full backs up "all files in the FileSet whether or not they have changed". An incremental backs up files changed since the last successful backup of the same job. A differential backs up files "that have changed since the last successful Full backup of the same Job" (Bareos Director configuration, Level).

Microsoft describes the same idea for SQL Server. A differential is "based on the most recent, previous full data backup" and captures "only the data that has changed since that full backup", and the full it is based on is called the base (Microsoft Learn, Differential backups (SQL Server)).

What does a restore actually need?

Example (illustrative): a file server fs01 is backed up to backup01. A full backup of 500 GB runs on Sunday. From Monday to Saturday, 20 GB of different files changes each day. Assume no file changes twice, which keeps the arithmetic simple.

Night Incremental stores Differential stores
Monday 20 GB 20 GB
Tuesday 20 GB 40 GB
Wednesday 20 GB 60 GB
Thursday 20 GB 80 GB
Friday 20 GB 100 GB
Saturday 20 GB 120 GB
Week total (including the 500 GB full) 500 + 6 x 20 = 620 GB 500 + (20+40+60+80+100+120) = 500 + 420 = 920 GB

To restore Saturday's state:

  • With incrementals you need the Sunday full and all six incrementals, applied in order: seven pieces.
  • With differentials you need the Sunday full and Saturday's differential: two pieces.

The differential week stores 300 GB more (920 - 620) and buys a shorter chain. A file that changes on several days is counted once in a differential, so real differentials grow more slowly than this, and real incrementals vary night to night. Treat the figures as a shape, not a prediction.

Microsoft states the SQL Server rule precisely: "before you restore a differential backup, you must restore its base. Then, restore only the most recent differential backup". The same page warns that "the longer the time between a differential backup and its base, the larger the differential backup is likely to be", and that differentials "can eventually approach the differential base in size".

Why do incremental and differential backups exist at all?

Because a full backup every night costs the most of everything: source reads, network, backup storage and time. Veeam describes an active full backup as retrieving "data for the whole VM from the source", and notes that such backups "are resource-intensive and consume a considerable amount of network bandwidth" (Active Full Backup; Synthetic Full Backup). Microsoft says a differential "can be much faster than creating a full backup".

The price is restore complexity. Microsoft: restoring from a differential "will necessarily take more steps and time than restoring from a full backup because two backup files are required".

What can an incremental or differential miss?

How "changed" is detected matters as much as the type. Bareos explains that its client "decides which files to backup for an Incremental backup by comparing start time of the prior Job" against each file's modified and attribute-changed times. Three consequences follow from its documentation:

  • Moved directories can be skipped. "If you move a directory rather than copy it, the files in it do not have their modification time ... changed", so they "will probably not be backed up by an Incremental or Differential backup".
  • Deleted files can come back. Any file deleted since the last full "remains in the Bareos catalog", so a restore can bring back files you deleted, until the next full. Bareos offers an Accurate mode to handle deletions and calls it "quite memory consuming".
  • A changed selection means a new full. Bareos treats any change to the file selection (a folder added or excluded) as a different FileSet, and when it cannot find a valid previous full it "will upgrade the Incremental to a Full save". One cause of an unexpectedly large "incremental" is this.

Other products detect change by blocks rather than file times. SQL Server uses a bitmap of changed extents (groups of eight pages), and Veeam copies "only VM data blocks that have changed since the last backup job session" (Forward Incremental Backup). The failure modes differ, and the label alone does not tell you which detection method is in use.

Why would anyone choose a differential?

Bareos gives its reason in plain words. A differential "effectively merges all the Incremental and Differential backups since the last Full backup into a single Differential backup". It lists two effects: redundancy, because "the old backups could be used if the merged backup cannot be read", and fewer storage volumes needed for a restore, "effectively eliminating the need to read all the volumes on which the preceding Incremental and Differential backups since the last Full are done".

In short, a differential costs extra storage each night in exchange for a shorter, less fragile chain.

How do I read a provider's label?

Ask these questions and get the answers in writing:

Question Why it matters
Is each night incremental or differential, and relative to what? Decides how many pieces a restore needs
How often is a new full made, and does the schedule enforce it? Veeam notes that it "does not create full backups automatically based on retention limits"; the administrator schedules them
Are deleted files removed from a restored copy, or do they return? Bareos documents that they can return without Accurate mode
What happens to the chain when the full is deleted? Veeam: "If you delete a full backup file, the entire chain of incremental backup files will become useless"
Is the whole chain checked, or only the newest piece? A chain is only as good as its weakest piece

How can I check this on my own system or with my provider?

  1. Ask for the list of pieces for one chosen restore date. Name a date, such as last Wednesday. The provider should name the full and every incremental or differential that date depends on.
  2. Find the reference point of the latest backup. On Bareos, the job report of a differential shows the start time it compared against after the word "Since". Elsewhere, ask the provider which backup the latest one is relative to.
  3. Check what the retention rule does with the chain. Veeam's forward incremental documentation shows that on some days more restore points exist on disk than the retention setting says, because the old full cannot go until the last increment that depends on it expires (Forward Incremental Backup Retention Policy).
  4. Restore one chosen date to a scratch location and compare it with the source. A completed job is not the same as a restore that works.
  5. For SQL Server, keep ad hoc backups out of the chain. Microsoft notes that a copy-only backup does not update the differential bitmap, so it "doesn't affect subsequent differential backups".

Risks and limits

  • Risk: deleting a full backup, or any piece in the middle of a chain, can make every later incremental unusable. Veeam documents this for forward incremental chains, and the same dependency follows by logic for any scheme in which later pieces build on earlier ones. There is no way back once the files are gone, unless another copy exists.
  • Each product uses these words in its own way. Check the product's own definition before relying on the general description above.
  • The 500 GB example assumes that no file changes twice and that all figures are decimal gigabytes. Real change patterns differ.
  • Vendor documentation pages describe each product's current behaviour; older versions can differ. Bareos pages show no version, so confirm behaviour on the version you run.
  • A backup type says how data is selected, not how it is stored. Compression, de-duplication and storage format change the real space used.

NetLeverage includes a daily full disaster-recovery backup of each hosted server at no extra charge. Talk to an engineer.

Sources