Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

Finding a bad backup before the day you need it

Technical article
Backup & Recovery

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

Check the copy you would actually retrieve, more than once, and decide in advance what a failed check means. A matching cryptographic hash against a trusted, protected reference, or a valid signature checked with a trusted key, provides evidence that the covered bytes have not changed. Opening a sample tells you a person or a program can still interpret those bytes. A backup job that finished tells you neither.

Do the checks before an incident, on a frequency you have written down. NIST SP 800-53 control CP-9(1) exists because retrieval and integrity are not guaranteed by the act of writing the backup. The Information Security Manual's digital preservation control exists because media, hardware and software age. A check you postponed until the outage is a check you may not have time to act on.

No backup was hashed or restored for this article. No hash algorithm is recommended by name, because the cryptographic publications that SP 800-53 cites for signatures were not opened in this session.

Why it matters

The controls discussed here apply where your organisation has selected or is required to meet them. They are not a blanket legal requirement for every business.

CP-9 requires the organisation to protect the confidentiality, integrity and availability of backup information. The discussion names the integrity mechanisms in one sentence: digital signatures and cryptographic hashes. That is a protection during the life of the copy. CP-9(1) is the test of that protection and of the media: test backup information at an organisation-defined frequency to verify media reliability and information integrity. The discussion splits reliability into three parts that fail separately. The systems that store the backup can fail. The operation used to retrieve it can fail. The information inside can fail. A hash that matched on the day of the write speaks to the third part only at that moment, and only if you hashed the information you care about. It does not speak to a disk that dies next month, or to a retrieve operation that cannot find the key.

SP 800-34 says backup copies should be tested regularly so that data are being stored correctly and files can be retrieved without errors or lost data. It also says each type of medium has a use and a storage life, after which the media cannot be relied on for effective recovery. Integrity is not only an attacker problem. It is a materials problem. The September 2026 manual's control ISM-1510 requires a digital preservation policy, and the paragraph above it names data degradation and the obsolescence of removable media, hardware and software as reasons the long-term integrity and availability of data need a policy. A hash algorithm you can no longer run, or a backup format whose software you uninstalled, is an obsolescence failure even when the bits are pristine.

SP 800-53 control SI-7 is the broader integrity control, not a backup-only control. It says to employ integrity verification tools to detect unauthorised changes to software, firmware and information the organisation lists, and to take defined actions when those changes are detected. The discussion lists mechanisms that can watch integrity: parity checks, cyclical redundancy checks, cryptographic hashes, and associated tools. Those mechanisms are not equal. A parity bit and a cryptographic hash do not catch the same class of change. SI-7(1) adds the timing: perform the check at startup, at defined transitional states or security-relevant events, or at a defined frequency. A backup repository is information at rest. If you have put it inside SI-7's scope, the frequency has to be one of those choices, not "when we remember."

ISM-1515 still sits above all of these checks. Restoration of data, applications and settings to a common point in time, tested in disaster recovery exercises, is the check that the information is not merely unchanged but usable as a set. A perfect hash of each member does not prove the set is from one point in time.

How it works

Seal the bytes, then check the seal on read-back

When the backup is written, record a cryptographic hash or a signature over a scope you can name. CP-9's discussion allows both and does not, in the text opened here, require a particular algorithm. Use a currently approved cryptographic mechanism under your organisation's security policy, document its scope, and protect the reference hash or verification key separately from anyone able to alter the backup. A hash stored beside the data is not tamper protection if an attacker can replace both. The older FIPS references in this edition of SP 800-53 are not a current algorithm approval list. Confirm current requirements with your security owner; this article does not select an algorithm.

The scope of the seal is the whole design. A hash of a filename list does not seal the file contents. A hash of ciphertext seals the ciphertext. CP-9(1)'s example of a real test decrypts a sample and compares it with known information, which means the integrity of the content, not only of the encrypted blob, is in scope for that example. If your seal is only over ciphertext, write that limit down. You have detected a changed blob. You have not detected a blob that was wrong before it was encrypted.

Check the seal when the copy is read back from the store you would use in an incident, not only on the server that computed it. A hash that never leaves the host that created the backup will match even if the offsite object is truncated, because you are not hashing the offsite object. CP-9(5) is the transfer to the alternate site. The seal has to be checked after that transfer if the transfer is part of the recovery path. Physical shipment is one of the transfer methods the control's discussion allows. Media that travelled can fail the seal even when the source passed it. That is the check doing its job.

SI-7's list is a reminder not to pretend every "checksum" is the same tool. A cyclical redundancy check, which the control names, is good at catching many accidental bit errors and is not a statement about who produced the data. A cryptographic hash, as CP-9 uses the term alongside digital signatures, is what you use when you need to detect change since a value was recorded. A signature adds a claim about who sealed it, which a bare hash does not, provided you trust the key. CP-9(8) says cryptographic protection of backups has to include thinking about key management. A signature whose trusted verification key is unavailable cannot presently supply the verification evidence you need.

Open a sample, because unchanged and usable are different

CP-9(1) says independent tests can cover each aspect of reliability. The discussion's example is: decrypt, transport a random sample from the alternate or backup site, and compare it with the same information at the primary site. Comparison catches silent change and catches a retrieve path that does not work. It requires the primary copy to still exist. After the primary site is gone, that comparison is unavailable, which is why you do it beforehand.

CP-9(2) goes further for selected functions: restore them from a sample of backup information during contingency plan testing, and choose the sample size from the assurance you need. An application that rejects a file will not be caught by a hash of a file it would also have rejected on the day of the backup. The hash sealed a bad file faithfully. The functional sample is the check for that class. SP 800-84's sample validation text lists "verify data integrity checking" and "verify system restoration" as separate lines. Keep them separate in the record.

What to sample is a risk decision, not a hope that randomness will be kind. The control discussion offers a random sample as an example. If you cannot do random, rotate across the systems whose loss hurts, and say you rotated. Always sampling the same tiny file trains the check to ignore everything else.

SP 800-34's description of incremental and differential backups adds a chain problem that looks like corruption and is sometimes just incompleteness. An incremental copy can be intact and still unrestorable if an earlier full backup or an earlier incremental is missing or unreadable. A hash of tonight's incremental can pass. The restore fails. Check that every required ancestor is present, integrity-verified and readable, then perform a supported restore to test the assembled chain. Presence and individual readability alone do not prove that generations can be combined correctly. SP 800-34 is explicit that incremental recovery can need the full backup and each later incremental, and that differential recovery can need the full backup and the last differential.

Decide the action when a check fails

SI-7 does not stop at detection. It requires defined actions when an unauthorised change is detected. For backups, write the actions before the first failure. Otherwise the failure will be debated as a tooling glitch until the incident.

A failed integrity check means the copy is not yet trusted. Preserve it and the original reference, quarantine it from normal recovery selection, and investigate the data, reference, keys and verification tool. Do not delete or overwrite the only evidence. Evaluate another generation using both integrity and functional checks, and record its recovery point. If every recent generation fails, possible causes include repository, transfer, reference or verification failures, which is the "systems where the backup is stored" part of CP-9(1), and it is a contingency-plan finding under CP-4: review the result and start corrective action.

A passed trusted integrity check and a failed open narrows the investigation to content validity, keys, dependencies, permissions, reader compatibility and the recovery environment; it does not identify a single cause. Causes include a backup taken of data the application cannot use, a missing key, or software you no longer have. ISM-1510's obsolescence point belongs here. The corrective action may be a new backup method, a retained copy of the reader software, or a change to what you seal. It is not a re-run of the same hash.

A passed sample and failed group restore can expose an unsampled fault, inconsistent recovery points, missing dependencies or an environment problem. Diagnose the failure and expand the test to cover the affected requirement. CP-9(2) ties the size to the assurance you need. A failure is new information about that assurance.

Do not "fix" a failed seal by recomputing the hash and storing the new value over the old one without a record. That erases the detection SI-7 asked for. If you re-seal after an investigated, legitimate rewrite, record who did it and why. Silent re-sealing is how corruption becomes the new baseline.

What to choose and why

Choose a seal on the object you would ship or replicate, and verify it after it arrives. For intentional alteration, use a signature with a trusted key or another approved mechanism with an independently protected integrity reference; a bare replaceable hash is insufficient. SI-7 lists parity and cyclical redundancy checks because they detect some faults. They are a weak sole control against someone who can rewrite the file and the parity together. The article does not claim they are useless for media errors. It claims they are not the same as the mechanisms CP-9 names for backup integrity.

Choose a functional sample on a schedule, from the offsite or alternate copy, for the functions whose loss is the reason you back up at all. CP-9(1) leaves the frequency to you. Tie that frequency to the storage life SP 800-34 mentions: if the medium's reliable life is measured in years, a single test at purchase does not cover the life. If the medium is a disk array that is scrubbed by the vendor's own tools, still run your seal and your open. The vendor scrub is not your application.

Choose to test chains at least as often as you test single full backups. A policy of weekly incrementals and a monthly hash of the latest incremental, with the full backup never opened, does not match SP 800-34's own description of what a restore requires.

Choose actions that a tired person can follow. "Notify the backup owner on the first failure; preserve and quarantine the suspect copy; verify the checker and trusted reference; test a suitable alternate generation; record the finding and escalate according to impact." That is enough to start. Refine it when CP-4's review shows a case the sentence did not cover.

Match depth to impact using SP 800-34 Table 3-6 as a US federal ladder, not as Australian law. That table marks backup reliability and integrity testing not applicable at low impact and applicable at moderate and high. ISM-1515, in the passages opened, does not offer that exemption. If you drop content tests because a US table says N/A, record the classification you are relying on, and do not also claim the Australian restoration control is met.

Risks and limits

The first risk is sealing the wrong thing and feeling finished. A hash of a catalogue entry, of a filename, or of ciphertext, filed as "integrity verified," will pass while the payload is unusable. Write the scope on the same line as the result.

The second risk is checking only the primary copy. CP-9(1)'s example transports the sample from the alternate or backup site. A match on the source host does not include the transfer.

The third risk is a sample that avoids the hard content. Opening a text file from a backup of a database is a text-file test. This article does not provide database restore procedures. It says the function in CP-9(2) has to be the function you would miss, or the assurance is about a different function.

The fourth risk is obsolescence disguised as a bit-perfect archive. ISM-1510 and SP 800-34's media-life note both say time and tool changes matter. A sealed file you cannot read is a preserved failure.

The fifth risk is naming an algorithm this research did not verify. SP 800-53 points at FIPS publications for cryptography. Those texts were not opened. Do not take any algorithm mentioned in a product brochure as if this article had endorsed it. Use the algorithm your cryptographic standard owner has already accepted, and record which one you used so the check can be repeated.

SI-7 is about detecting unauthorised change to software, firmware and information. It is not a statement that every integrity tool understands backup formats. Pointing a file-integrity watcher at a live disk does not watch the offsite backup unless you aimed it there. Aim it, or do not claim SI-7 coverage of the backups.

This article was written with AI assistance. It is source research only. The factual claims have been independently checked against the cited sources. No integrity tool was run.

Sources