Kubernetes Persistent Data: CSI Snapshots, Velero and Application Consistency

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

A CSI snapshot does not by itself guarantee application consistency, and ordinarily covers one volume unless a supported group snapshot is used. It does not coordinate a database's state, and it may not leave the storage system that holds the original. Velero can add the Kubernetes objects and move the data elsewhere, but application consistency is still your design. Prove it in an isolated restore environment; a separate namespace alone does not isolate network access, external services or cluster-wide resources.

The commands below were not run in a lab. Names are examples.

Recommendations and interpretations are editorial unless attributed; source research is not lab validation.

Why it matters

An etcd restore brings back the claims and pods, but the contents of those volumes live in the storage system. Stateful workloads need a second, separate protection path.

That path has three questions. Is the copy consistent? Is it somewhere that survives the failure? Can it be turned back into a working application? Snapshot tooling answers only part of each.

How it works

The snapshot API

Three API objects do the work, and all are custom resources rather than part of the core API:

Snapshots work only with CSI drivers, and a driver may or may not implement them. Installing the custom resource definitions and snapshot controller is the responsibility of the Kubernetes distribution. The controller creates and binds the content objects, and a csi-snapshotter sidecar beside the driver calls CreateSnapshot and DeleteSnapshot.

A minimal request:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: db-before-change
spec:
  volumeSnapshotClassName: csi-example-snapclass
  source:
    persistentVolumeClaimName: db-data

A restore creates a new claim with a dataSource that names the snapshot:

spec:
  dataSource:
    name: db-before-change
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io

So the snapshot API creates a new volume from a snapshot. It does not roll the existing volume back.

Deletion policy

The class must set a deletionPolicy. With Delete, removing the VolumeSnapshot also deletes the storage snapshot. With Retain, both the content object and the storage snapshot remain.

So under Delete, an accidental namespace deletion can take the snapshots stored in that namespace with it. Take care over who can delete these objects. For plain Velero CSI snapshots, Velero's answer is a Retain class, which keeps the storage snapshot for the lifetime of the backup, even if the namespace holding the VolumeSnapshot is lost.

What readyToUse means

The CSI specification separates cutting a snapshot from finishing it. A driver that uploads or processes afterwards returns ready_to_use as false after the cut, and it becomes true when the snapshot can create new volumes.

The specification also says the orchestrator may freeze the application before the snapshot, and may thaw it once the snapshot is cut. The Kubernetes snapshot objects documented above have no field that requests a freeze. Nothing freezes your application unless you arrange it.

Consistency levels

Be exact about which of these you have (the tiers are our framing):

The freeze row needs a qualifier. The man page says fsfreeze is intended for hardware RAID devices that support snapshots, and is unnecessary for device-mapper devices, where a snapshot request freezes the filesystem automatically. Whether a freeze adds anything depends on what sits under your CSI driver.

Velero's own comparison lists its CSI-based data movers as point-in-time and crash-consistent, and File System Backup as not point-in-time.

Crash-consistent can be a good result. PostgreSQL says a frozen volume snapshot taken while the server runs is usable. On start the server believes it crashed and replays its write-ahead log, which is not a problem as long as those log files are included. PostgreSQL conditions this on the file system supporting it "and you are willing to trust that it is implemented correctly". A CSI driver's snapshot is exactly what that warning covers.

More than one volume

That guarantee breaks when an application spreads across volumes. PostgreSQL warns that with data files and WAL on different disks, the snapshots must be simultaneous. Snapshots taken one after another do not provide that.

The CSI answer is the volume group snapshot, which must give a write-order consistency guarantee or fail. It needs driver support and its own custom resources, and the served API is v1beta2. The project's sources differ on maturity: the sidecar's README calls the feature gate GA, while the CSI developer guide lists it as beta from Kubernetes 1.32. Check your own driver and sidecar release.

Velero has supported group snapshots since 1.17, and its default is still one snapshot per claim, with no atomic consistency across volumes. You opt in with a label. Claims in the same namespace with the same velero.io/volume-group value are snapshotted together:

kubectl label pvc -n app db-data db-wal velero.io/volume-group=db

With --snapshot-move-data, Velero creates the group snapshot for write-order consistency, then moves each volume's data and cleans up. Without it, the snapshots stay on the storage system. You need the custom resources, driver support and a VolumeGroupSnapshotClass labelled velero.io/csi-volumegroupsnapshot-class: "true". Velero detects the served API version at run time. A restore needs no group snapshot logic, and can go to a cluster without group snapshot support. The consistency still depends on your driver.

What Velero adds

Velero uploads a tarball of Kubernetes objects to object storage and handles volume data separately. It warns that cluster backups are not strictly atomic, so objects edited during a backup might not be captured.

There are three main routes for volume data:

Velero's pages conflict on hostPath: the data movement comparison lists it for File System Backup, while the File System Backup page says hostPath volumes are not supported. We follow the second.

The first route has a catch that decides many designs. Velero notes that only Kubernetes objects are uploaded, not snapshot data, and that not all drivers guarantee durability, so a snapshot may be exposed to the same loss as the original volume. If the array or pool fails, those snapshots may go with it.

What to choose and why

The following are our recommendations, based on the limits above:

Velero says data movement should be preferred whenever CSI snapshots are available, because File System Backup reads the live volume.

Velero offers two built-in data movers. velero-fs is the default and suits file storage. velero-block reads raw blocks on block storage and can use the CSI changed block tracking API for incrementals, and it runs only on Linux nodes. File-backed volumes cannot use it, changed block tracking needs CSI driver and storage platform support, and that service may require a configured ServiceAccount. Data movement as a whole requires the Kubernetes MountPropagation feature.

velero install --features=EnableCSI --use-node-agent \
  --plugins=<object storage plugin> ...

velero backup create app-daily --include-namespaces app \
  --snapshot-move-data --data-mover velero-fs

File System Backup deserves extra care. It needs node agent pods running as root, and privileged in some environments. Large files such as database files take a long time to scan, even when little has changed.

Two timeouts apply. A CSI snapshot has 10 minutes by default to become ready and the asynchronous data movement 4 hours, and an upload that overruns is cancelled. Set them from measured times on your largest volume.

Velero's built-in movers and File System Backup use a static, common encryption key for all repositories, so anyone with access to the backup storage can decrypt the data. Restrict that access tightly, and add bucket-level controls of your own.

Getting closer to application consistency

Velero backup hooks run commands in pod containers before and after the pod is backed up. They are set with annotations. Key defaults:

• Commands do not run in a shell unless you start one.
• The default timeout is 30 seconds.
• A non-zero exit defaults to Fail. Continue is the other option, and it produces a backup you should not treat as quiesced.
• Since 1.15, hooks run around an ItemBlock. Pods sharing an RWX volume all run their pre hooks, then the items are backed up, then all post hooks run.

Velero documents filesystem-freeze hooks using a dedicated sidecar with access to the same volume. Treat this as a design requiring a measured pause budget, an independent thaw mechanism and a tested failure path; a normal application pod does not automatically provide that sidecar. A pre-hook success must be paired with evidence that post-hook or emergency thaw completed.

Warning: a frozen filesystem blocks every process that tries to write until it is unfrozen. If the post hook does not run, the application stays stuck. Test failure paths and have a manual unfreeze ready. Check that your pod security rules allow the freeze before relying on it. Writes stay blocked until the post hook runs, so this is an application pause. The 30-second hook timeout applies to the command, not to how long the freeze lasts.

The freeze makes the filesystem stable. It says nothing about what the application still holds in memory, so it does not make the database's own state safe.

A database hook needs more thought, because many database backup modes depend on an open session. PostgreSQL's low-level API requires that the connection calling pg_backup_start stays open until the end of the backup, or the backup aborts. A hook is a single command that exits, so the connection would close and the backup would abort. That API is not a match for a simple pre and post hook.

Velero's page uses MySQL's FLUSH TABLES WITH READ LOCK to illustrate environment variables in a hook. It runs through mysql -e, a client that exits as soon as the statement finishes, so read it as an example of environment variables, not as a working quiesce. MySQL's locking documentation states that session termination releases table locks. The FLUSH statement uses a global read lock, a distinct mechanism. Those excerpts do not validate a lock spanning separate short-lived hook clients; use a database-supported coordination process that proves the protection remains active through capture.

For PostgreSQL on one volume, the practical options are a crash-consistent snapshot with a CHECKPOINT beforehand to reduce recovery time, or a continuous archiving base backup, which is immune to filesystem changes. The second also gives point-in-time recovery, so it should remain the primary method for important data. Where several volumes cannot be snapshotted together, PostgreSQL also offers to shut down the server long enough to establish all the snapshots, which suits a small maintenance window.

Restoring and proving it

Velero restores in a fixed order: custom resource definitions and namespaces first, volume snapshot objects before PersistentVolumes and claims, and pods after those. For a CSI backup, it sets the claim's dataSource to the VolumeSnapshot, so the driver creates the volume.

Behaviours that catch people out:

• Restores are non-destructive by default. Velero skips anything that already exists, so a broken application with an existing claim stays as it is. --existing-resource-policy=update is best-effort and falls back to a warning.
• Replacing data in existing claims is a separate option. Use --existing-volume-data-policy=full or incremental and remove consuming pods first. For File System Backup, delete the workload controllers rather than scaling to zero, because a scaled-down controller kills the pod holding the restore-wait container.
• API versions must exist on the target. The backed-up group and version must be served by the target cluster.
• Status is removed by default. Velero strips the status field unless told to keep it, which matters for some custom resources.

Protect the repository during a real recovery. Velero documents setting the backup location to read-only:

kubectl patch backupstoragelocation <name> --namespace velero \
  --type merge --patch '{"spec":{"accessMode":"ReadOnly"}}'

When ready, set it back to ReadWrite with the same command, or backups will not resume.

Evidence that a backup ran is not evidence that it restores. Check each layer:

velero backup describe app-daily --details
velero backup logs app-daily | grep hookCommand
kubectl get volumesnapshot -n app   # plain CSI route only
kubectl -n velero get datauploads -l velero.io/backup-name=app-daily   # data movement

describe reports HooksAttempted and HooksFailed. For data movement, confirm the DataUpload reached Completed. Expect no snapshot objects after that route, because Velero deletes the snapshot once the data has moved.

Then restore into a separate namespace with --namespace-mappings:

velero restore create app-test --from-backup app-daily \
  --namespace-mappings app:app-restore-test
velero restore describe app-test --details

The test has passed when the application itself agrees. Start the database, let recovery finish, and run its own integrity check. Compare row counts or checksums with known values, and record how long it took. Restore hooks can automate parts of this: init container hooks run setup before the application containers start, and exec hooks run commands in the containers of the restored pod. Make sure the test copy cannot reach production services.

Risks and limits

• Retention deletes real data. Velero's default TTL is 30 days, and expiry removes the backup and its volume snapshots.
• Retain does not outlive the backup. Velero's CSI snapshots are kept only for the life of the backup, even with a Retain class. After data movement Velero deletes the snapshot regardless, and it recommends Delete for the built-in movers. With Retain, a data movement that fails or is cancelled before the handoff may leave the snapshot behind for manual cleanup.
• Object storage is the source of truth. A Completed backup missing from storage is removed from Kubernetes. Protect the bucket accordingly.
• Default classes must be unique. More than one default VolumeSnapshotClass per driver makes snapshot creation fail.
• Not covered: vendor storage operators, database operators with their own backup systems, and individual driver behaviour. Confirm these in the vendor's documentation.

Sources

Opened 2026-11-10.

• Kubernetes: Volume snapshots, Volume snapshot classes, Persistent volumes
• CSI specification, external-snapshotter README, CSI developer guide: group snapshots
• Velero v1.19: How Velero works, CSI support, CSI snapshot data movement, File System Backup, Backup hooks, Restore hooks, Restore reference, Backup API, Disaster recovery
• fsfreeze(8), PostgreSQL: File system level backup, PostgreSQL: Continuous archiving
• Velero: Volume Group Snapshots, Velero 1.17 changelog

This article was originally published on 10 November 2026.