Australian cloud for business software — built and run in Sydney since 2002

NetLeverage
NetLeverage

Kubernetes "Multi-Attach error for volume": "Volume is already used by pod(s)"

Technical article
Troubleshooting

About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.

A pod that cannot start because of the event Multi-Attach error for volume "pvc-..." Volume is already used by pod(s) web-6d8f-x7k2p is waiting for a volume that is still attached to a different node. The volume allows one node at a time. In Kubernetes v1.37 the same event begins "Waiting for detach for volume" instead. The wait ends only when the pod holding the volume is gone and the volume has detached.

What the error means

The Kubernetes persistent volumes page defines ReadWriteOnce as "the volume can be mounted as read-write by a single node", and adds that it "still can allow multiple pods to access (read from or write to) that volume when the pods are running on the same node." ReadWriteOncePod limits access to one pod in the cluster.

The attach/detach reconciler in the Kubernetes source checks whether a volume may attach to more than one node. A persistent volume whose access modes include ReadWriteMany or ReadOnlyMany may, and so may one that lists no access mode. A persistent volume with only other access modes, such as ReadWriteOnce, may not. If it is already attached to another node, the reconciler records a Warning event with reason FailedAttachVolume on the pod that is waiting.

The wording depends on the release, and it comes from Kubernetes' source code, not from a documentation page. The source builds each event from a format string, "%v for volume %q %v": the first part is the prefix, then the volume name, then the reason text. In the release tags checked (v1.28.0, v1.31.0, v1.33.0, v1.35.0 and v1.36.0) the prefix is "Multi-Attach error". In tag v1.37.0 it is "Waiting for detach". For a claim-backed volume, the name in quotes is the persistent volume's name. Two forms exist in each release:

Release When the reconciler still sees pods using the volume When it sees none
v1.36.0 and earlier checked Multi-Attach error for volume "pvc-..." Volume is already used by pod(s) NAME Multi-Attach error for volume "pvc-..." Volume is already exclusively attached to one node and can't be attached to another
v1.37.0 Waiting for detach for volume "pvc-..." Volume is already used by pod(s) NAME Waiting for detach for volume "pvc-..." Volume is already exclusively attached to one node, waiting on detach before it can be attached to another node

The pod list can end with "and N pod(s) in different namespaces", or read "Volume is already used by N pod(s) in different namespaces" when no pod in your namespace uses it.

Likely causes

Cause What you see How to confirm
Rolling update puts the new pod on another node Old pod Running on one node, new pod waiting on another; the message names the old pod kubectl get pods -o wide shows two nodes; strategy is RollingUpdate
More than one replica shares one ReadWriteOnce claim Some replicas run, the rest wait with the message kubectl get pvc shows access mode RWO; replicas above 1
Node shut down or lost, volume not released The "exclusively attached" form; pods stuck Terminating on the dead node A VolumeAttachment still names the dead node; the node is NotReady
Old pod deleted, detach slow or failing The "exclusively attached" form, lasting status.detachError on the VolumeAttachment
Another namespace uses the same volume "pod(s) in different namespaces" kubectl get pods -A -o wide

Prove the cause before changing anything

All of these only read.

  1. Read the events of the waiting pod:
kubectl events --for pod/web-7b5d9-new -n shop
  1. See which node each pod runs on:
kubectl get pods -n shop -o wide
  1. Check the claim's access mode and volume. The persistent volumes page abbreviates ReadWriteOnce as RWO in the CLI:
kubectl get pvc -n shop
kubectl get pv
  1. See which node holds the volume. The VolumeAttachment API object names the node in spec.nodeName and records success in status.attached and errors in status.attachError and status.detachError:
kubectl get volumeattachments
kubectl get volumeattachments -o yaml
  1. Check the node and the update strategy:
kubectl describe node k8s-node2
kubectl get deployment web -n shop -o yaml

A good result for a normal rolling update is that the old pod is Terminating and the VolumeAttachment no longer lists the old node. A bad result is a healthy old pod that stays Running, or a VolumeAttachment that stays on a NotReady node.

Example (illustrative): Deployment web has one replica and a claim web-data with access mode RWO, bound to volume pvc-8f1c2d, attached to k8s-node1. An image update creates web-7b5d9-new, which the scheduler places on k8s-node2. Its event reads Multi-Attach error for volume "pvc-8f1c2d" Volume is already used by pod(s) web-6d8f-x7k2p. The defaults are maxSurge 25% (rounded up) and maxUnavailable 25% (rounded down). For 1 replica, surge is 0.25 rounded up to 1, so one extra pod is allowed. Unavailable is 0.25 rounded down to 0, so the number of available pods must stay at 1 - 0 = 1. The old pod therefore stays until the new pod is available, and the new pod cannot start until the old one releases the volume. The update cannot finish by itself.

Fix

Rolling update on one claim: use Recreate

The Deployment documentation says "All existing Pods are killed before new ones are created when .spec.strategy.type==Recreate." That ends the deadlock in the example, because the old pod ends first.

First save the current manifest:

cp -p web-deployment.yaml web-deployment.yaml.2026-11-22

Risk: the next change makes every later update of this Deployment end all its pods before new ones start, so each update has downtime. The Deployments page says "All existing Pods are killed before new ones are created when .spec.strategy.type==Recreate." On a healthy Deployment nothing restarts when you apply it, because a strategy change alone is not a pod template change.

In web-deployment.yaml set strategy to type: Recreate and remove the rollingUpdate block, because maxSurge and maxUnavailable belong to RollingUpdate. Check it on the server, then apply it:

kubectl apply --dry-run=server -f web-deployment.yaml
kubectl apply -f web-deployment.yaml

To undo it: put the saved copy back under its original name, check it, then apply it:

cp -p web-deployment.yaml.2026-11-22 web-deployment.yaml
kubectl apply --dry-run=server -f web-deployment.yaml
kubectl apply -f web-deployment.yaml

Several replicas on one ReadWriteOnce claim

Recreate does not fix this cause, because all replicas still need the same claim on different nodes. The remedies are a design change: one replica, a volume type with ReadWriteMany access, or a separate claim per replica. A Kubernetes source comment describes this case as typically a user error, such as a ReplicaSet with more than one replica using one PVC. Each of these changes the application's design and belongs in your normal change process.

A node that is shut down and holds the volume

The node shutdown page explains that when a node shuts down and the kubelet does not detect it, StatefulSet pods stay Terminating, "the VolumeAttachments will not be deleted from the original shutdown node so the volumes used by these pods cannot be attached to a new running node." If the node comes back up, the pods are deleted and recreated elsewhere. If it never does, the page's remedy is the node.kubernetes.io/out-of-service taint.

Before you taint, confirm outside Kubernetes (in your hypervisor or cloud console) that k8s-node2 is shut down or powered off and not restarting. The page requires this, and says "Deviation from the steps documented above can result in data corruption."

First save the node's current state:

kubectl describe node k8s-node2 > k8s-node2-describe-2026-11-22.txt
kubectl get pods -A -o wide > pods-2026-11-22.txt

Disruptive step: the next command marks k8s-node2 out of service. Pods on it without a matching toleration are forcefully deleted, and volume detach for them happens immediately. If the machine is still running a workload that uses the volume, that data can be damaged.

Add the taint, using the key and effect from the taints page and the key-only form from the kubectl taint page:

kubectl taint nodes k8s-node2 node.kubernetes.io/out-of-service:NoExecute

To undo it: remove the taint, but only after the pods run on another node and you have checked that k8s-node2 has recovered, which is when the page requires it:

kubectl taint nodes k8s-node2 node.kubernetes.io/out-of-service:NoExecute-

Do not delete VolumeAttachment objects or force-delete pods by hand. Neither can be undone. The shutdown page also says that where a pod deletion has not succeeded for 6 minutes, Kubernetes force-detaches volumes on an unhealthy node, with a risk of data corruption for any workload still using the volume.

Confirm the fix

  1. kubectl events --for pod/web-7b5d9-new -n shop shows the attach succeeding and no new FailedAttachVolume event.
  2. kubectl get pods -n shop -o wide shows the new pod Running.
  3. kubectl get volumeattachments lists the volume against the node where the new pod runs only.
  4. After the next update, the Deployment finishes its rollout without waiting.

What to watch afterwards

  • Any workload with a ReadWriteOnce claim and a RollingUpdate strategy: it can deadlock on the next update.
  • Replica counts above 1 on a Deployment that uses one claim.
  • Nodes that go NotReady while pods with volumes are on them.
  • The status.detachError field of VolumeAttachment objects.

Risks and limits

  • These commands have not been run or lab-tested.
  • Message wording is taken from Kubernetes' source code at the release tags named in the version scope. Other releases between those tags were not checked.
  • A taint on a node that is not actually powered off can corrupt data. The shutdown page says to check first.
  • CSI drivers can add their own attach and detach errors. Read the driver's documentation for those.

Sources