About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.
If versioning is switched on for the bucket, deleting a backup file does not remove it. S3 adds a delete marker, the file becomes a noncurrent version, and you keep paying for its full size. A backup tool that "deletes expired backups" every night therefore frees nothing. The bucket only shrinks when a lifecycle rule permanently expires noncurrent versions.
The delete marker itself is almost free. The cost is the hidden copies underneath it. Measure them first, then choose how long you want deleted data to stay recoverable.
What the symptom means
AWS describes versioning as keeping multiple variants of an object in one bucket. In a versioning-enabled bucket, "if you delete an object, Amazon S3 inserts a delete marker instead of removing the object permanently. The delete marker becomes the current object version." An overwrite creates a new version instead of replacing the old one.
Two sentences from the S3 User Guide explain the bill. First: "Each version of an object is the entire object; it is not just a diff from the previous version. Thus, if you have three versions of an object stored, you are charged for three objects." Second, on delete markers: "A delete marker doesn't have data associated with it", and the storage charged for one "is equal to the size of the key name".
Two things follow:
- Ordinary listings hide the problem. A
ListObjectsV2request does not return objects whose current version is a delete marker, so a file browser or the backup tool's own inventory looks empty or small while the bucket total stays high. Only aListObjectVersionsrequest (the console's Show versions switch) shows markers and old versions. - Once a bucket has been version-enabled it can never return to unversioned. You can only suspend versioning, which stops new versions accruing and leaves existing ones in place.
A rule that used to work can also stop working. The guide warns that if an unversioned bucket had an object-expiration rule and you later enable versioning, you "must add a noncurrent expiration configuration" to keep the same permanent-delete behaviour. An Expiration action on a versioned bucket only adds a delete marker.
Likely causes
| Cause | What you see | How to confirm |
|---|---|---|
| Backup tool deletes expired files with a plain delete | Bucket total keeps rising while the tool reports a steady number of files | list-object-versions shows DeleteMarkers entries and Versions entries with IsLatest false |
| Tool rewrites the same keys (index or manifest files) | Many versions of a few keys | Repeated Key values in Versions, each with a different VersionId |
Lifecycle has only Expiration, no NoncurrentVersionExpiration |
Rule exists and "works" on current files, but total never falls | Read the lifecycle configuration and look for the noncurrent action |
| Noncurrent rule exists but is set very long, or keeps many versions | Space returns slowly or only partly | NoncurrentDays and NewerNoncurrentVersions values; both must be exceeded before deletion |
| Rule disabled, or its prefix, tag or size filter does not match | Nothing expires | Status is Disabled, or the Filter excludes the keys the tool writes |
| Locked versions | Rule is correct but some versions stay | Object Lock retention or legal hold on those versions |
| Leftover incomplete multipart uploads | Bytes are counted but no object is visible | See the large-upload article; they need their own abort rule |
On locked versions, AWS states that lifecycle rules "continue to function normally on protected objects, including placing delete markers. However, a locked version of an object cannot be deleted by a S3 Lifecycle expiration policy."
Prove the cause before changing anything
These commands read only. They are taken from the AWS CLI reference and have not been run.
-
Measure, do not count. In the S3 console, S3 Storage Lens includes the free-tier metric
NonCurrentVersionStorageBytes("Number of noncurrent version bytes") andNonCurrentVersionObjectCount, plusDeleteMarkerObjectCount. CloudWatch publishesBucketSizeBytesonce a day and states that it covers "both current and noncurrent objects". Compare noncurrent bytes with total bytes. If noncurrent bytes are most of the total, you have found the growth. -
Sample the versions for one prefix. The call is paginated, so ask for a small page:
aws s3api list-object-versions --bucket backup01-bucket --prefix daily/ --max-items 50
Good result: entries under Versions with IsLatest true for files you expect, and few others. Bad result: many Versions entries with IsLatest false, or DeleteMarkers entries with IsLatest true, for files the backup tool believes it has removed. The reference lists Size for each version, so you can see how large the hidden ones are.
- Read the lifecycle configuration and save it to a file before you do anything else:
aws s3api get-bucket-lifecycle-configuration --bucket backup01-bucket
Look for NoncurrentVersionExpiration, the Status of each rule, and the Filter.
Example (illustrative): backup01 stores nightly files in a versioned bucket. The tool keeps 30 days and removes the expired 25 GB each night, so the live set is 30 x 25 = 750 GB. After a year the tool has uploaded 365 x 25 = 9,125 GB and removed 8,375 GB of it by plain delete. Every removed file is still a noncurrent version, so the bucket holds all 9,125 GB, about 12 times what the tool reports. With noncurrent versions expiring 7 days after they become noncurrent, steady state is about 750 + 7 x 25 = 925 GB (a little more, because expiry times round up to the next midnight UTC), roughly a tenth of the 9,125 GB.
Fix
Cause: no rule for noncurrent versions
Choose the recovery window first. The window is how long you can still undo a delete or an overwrite, whether by mistake or by an attacker. AWS gives the same reasoning: "Delayed removal of noncurrent objects can be helpful when you need to correct any accidental deletes or overwrites."
The lifecycle JSON below uses fields shown in the put-bucket-lifecycle-configuration reference. The 7 days is an illustration, not a recommendation.
{
"Rules": [
{
"ID": "expire-noncurrent-and-markers",
"Filter": { "Prefix": "" },
"Status": "Enabled",
"NoncurrentVersionExpiration": { "NoncurrentDays": 7 },
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
]
}
NoncurrentDays counts from the moment a newer version is created (the successor), rounded up to the next midnight UTC. ExpiredObjectDeleteMarker removes a marker once no noncurrent versions remain beneath it. The lifecycle documentation puts both actions in one rule in its example, and says the marker action cannot be used with a tag-based filter.
Disruptive step: applying the rule. put-bucket-lifecycle-configuration "will overwrite an existing lifecycle configuration", so merge your saved rules into the new file first. AWS also says a new configuration applies "to both existing objects and objects that you add", so the first run can permanently delete years of accumulated noncurrent versions. AWS describes the result of NoncurrentVersionExpiration as: "These deleted objects can't be recovered." There is no way back. To limit the blast radius, start with a narrow prefix, or keep a few versions with NewerNoncurrentVersions (1 to 100; the rule then needs a Filter element).
aws s3api put-bucket-lifecycle-configuration --bucket backup01-bucket --lifecycle-configuration file://lifecycle.json
Cause: tool rewrites the same keys
The same rule applies. If you need only the newest copy of those keys, a short window is enough.
Cause: long window or high version count
Both NoncurrentDays and NewerNoncurrentVersions must be exceeded before a version is deleted. Lower one, or both, and apply the same cautions.
Cause: disabled or mismatched rule
AWS: "If a rule is disabled, Amazon S3 doesn't perform any actions specified in the rule." Correct the Status or the Filter. A rule with a prefix filter will not touch keys outside the prefix.
Cause: locked versions
Wait for the retention to end, or change the design. The linked S3 Object Lock article explains what each mode allows. Lifecycle cannot remove a locked version.
Cause: incomplete multipart uploads
They need their own AbortIncompleteMultipartUpload rule, because AWS says "Object expiration lifecycle configurations don't remove incomplete multipart uploads." The large-upload article covers it.
Confirm the fix
- Allow time. AWS notes "there may be a delay between the expiration date and the date at which Amazon S3 removes an object", and that "you are not charged for expiration or the storage time associated with an object that has expired".
- Watch
NonCurrentVersionStorageBytesand the dailyBucketSizeBytesfall over the following days. - Re-run the version sample for a prefix. Old
IsLatestfalse entries older than your window should be gone. - Do a trial restore of a file the backup tool still holds, to prove the rule did not remove anything the tool needs.
What to watch afterwards
- Alert on noncurrent bytes as a share of total bytes, and on
DeleteMarkerObjectCount. - Keep the lifecycle JSON in version control. It is the only record of your recovery window.
- Limit who may change lifecycle rules. AWS says a bucket policy cannot stop a lifecycle rule from deleting: "even if your bucket policy denies all actions for all principals, your S3 Lifecycle configuration still functions as normal". The put-bucket-lifecycle-configuration reference lists
s3:PutLifecycleConfigurationamong the permissions to deny if you want to stop users removing objects. Someone who can write a short rule can erase the recovery window. - Re-check after any change to the backup tool's retention. A tool that deletes more often creates more versions.
Read next
Related articles you may want next.
- CloudS3 Object Lock and versioning: designing backups that an admin cannot deleteWhat versioning, MFA delete, legal holds, governance and compliance mode each stop, who can undo them, and a safe build order for a backup bucket.Read article
- CloudLarge Uploads Stall or Restart: Multipart, Resume and Leftover PartsLarge uploads restarting or leaving hidden S3 parts? Understand multipart resume, abort incomplete uploads, add a lifecycle clean-up rule, and resume SFTP and rsync safely.Read article
- Backup & RecoveryArchive storage tiers for backups: minimum durations, retrieval time and the recovery billHow AWS, Azure and Google archive tiers set minimum storage periods and retrieval delays, and what that means for backup recovery time and cost.Read article
Risks and limits
- These commands have not been run or lab-tested. They are taken from the AWS documentation.
- Permanent expiry is irreversible. A shorter window saves money and shortens the time in which you can undo a delete.
- Editing a lifecycle configuration created before September 2024 can change its default for small-object transitions: AWS states that if you "create, edit, or delete rules, the default transition behavior for your configuration changes to the updated behavior", which stops objects under 128 KB moving to another storage class. Check any transition rules you also run.
- This page covers an unlocked, versioned bucket. A locked bucket behaves differently, as described in the linked Object Lock article.
- The AWS pages show no dates, and AWS can change lifecycle behaviour. Compare the
put-bucket-lifecycle-configurationreference for your CLI version. - Other S3-compatible services may implement versioning and lifecycle differently from AWS.
Sources
- Retaining multiple versions of objects with S3 Versioning — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Working with delete markers — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Lifecycle configuration elements — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Examples of S3 Lifecycle configurations — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Expiring objects — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Managing the lifecycle of objects — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Managing object locks (Lifecycle with Object Lock) — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- S3 Storage Lens metrics glossary — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- Amazon S3 metrics and dimensions in CloudWatch — Amazon S3 User Guide, no date displayed, accessed 17 November 2026
- list-object-versions — AWS CLI command reference, no date displayed, accessed 17 November 2026
- get-bucket-lifecycle-configuration — AWS CLI 2.40.4 command reference, no date displayed, accessed 17 November 2026
- put-bucket-lifecycle-configuration — AWS CLI command reference, no date displayed, accessed 17 November 2026
This article was originally published on .