About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.
Client certificates that kubeadm generates expire after 1 year. When they lapse, kubectl on a kubeadm cluster can fail with error: You must be logged in to the server (Unauthorized), and the kubeadm documentation names x509: certificate has expired or is not yet valid as an error seen in the kube-apiserver logs when a kubelet's client certificate cannot rotate. Run kubeadm certs check-expiration on a control-plane node to see which certificates have lapsed. Then save /etc/kubernetes, renew with kubeadm certs renew all on every control-plane node, and restart the control-plane static Pods.
What the symptom means
The kubeadm certificate management page says "Client certificates generated by kubeadm expire after 1 year." The default validity is set in the kubeadm configuration (kubeadm.k8s.io/v1beta4): certificateValidityPeriod defaults to 8760h, which is 365 days (8760 ÷ 24 = 365), and caCertificateValidityPeriod defaults to 87600h, 10 years.
kubeadm renews all certificates during a control plane upgrade. The page says that if you upgrade regularly, with less than a year between upgrades, kubeadm keeps the cluster up to date. A cluster that was built once and never upgraded, or one where --certificate-renewal=false was passed, reaches the one-year mark with the old certificates.
The two messages come from different places. x509: certificate has expired or is not yet valid is a certificate validity failure. The kubeadm troubleshooting page ties that wording to one case: when a kubelet's client certificate rotation fails, such errors appear in the kube-apiserver logs. You must be logged in to the server (Unauthorized) is kubectl's own wording when the API server answers with an Unauthorized status. That text comes from kubectl's source code (release tag v1.34.0), which builds it from the format error: You must be logged in to the server (%s); the %s is the message the API server returns, Unauthorized in this case. Kubernetes documents no table of which expired certificate produces which message, so read the expiry dates rather than infer the cause from the wording.
The kubeadm certs check-expiration command covers the certificates in /etc/kubernetes/pki and the client certificates embedded in admin.conf, controller-manager.conf and scheduler.conf. The kubelet.conf file is not included, because kubeadm configures the kubelet for automatic renewal under /var/lib/kubelet/pki.
Likely causes
| Cause | What you see | How to confirm |
|---|---|---|
Control-plane certificates or admin.conf passed their one-year expiry |
kubectl commands fail from every machine that uses the old credentials, for example with the Unauthorized message | kubeadm certs check-expiration lists an EXPIRES date in the past |
| The cluster was not upgraded within a year, or renewal was switched off | Same, on a cluster that has not been upgraded for more than 12 months | Last upgrade date; whether --certificate-renewal=false was passed |
| A kubeadm node's kubelet client certificate rotation failed | A node does not become Ready; x509: certificate has expired or is not yet valid in kube-apiserver logs |
The kubeadm troubleshooting page section on kubelet client certificate rotation |
| A kubeconfig copy still holds the old administrator certificate | kubectl fails for one user while /etc/kubernetes/admin.conf works |
Compare with KUBECONFIG=/etc/kubernetes/admin.conf |
| Certificates are managed by an external CA | check-expiration shows EXTERNALLY MANAGED as yes | The last column of the check-expiration output |
| A different failure that looks similar: CA mismatch | x509: certificate signed by unknown authority |
The troubleshooting page treats this as a possible certificate mismatch, not an expiry |
Prove the cause before changing anything
All of these are read-only.
1. List expiry dates on a control-plane node.
kubeadm certs check-expiration
The output has columns for the certificate, EXPIRES, RESIDUAL TIME, the certificate authority and whether it is externally managed. Certificates and kubeconfig-embedded certificates are listed first, and the CAs after them. A good result is every EXPIRES date in the future. A bad result is any date that has passed. The page says kubeadm cannot manage certificates signed by an external CA.
2. Test with the administrator credentials on the control-plane node itself.
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl get nodes
unset KUBECONFIG
If this works while your own $HOME/.kube/config fails, only the copy is stale. The troubleshooting page uses these KUBECONFIG forms.
3. Look at the certificate inside a kubeconfig. The troubleshooting page says the certificates in a kubeconfig file are base64 encoded, that base64 --decode decodes the certificate, and that openssl x509 -text -noout shows its details, including the validity dates.
4. Read the API server's log if the API is up enough to start. On a Linux node with a CRI container runtime, the crictl page documents listing containers with crictl ps -a and reading a container's logs with crictl logs --tail=N <container-id>, where the page's own example uses --tail=1 to show the latest line only:
crictl ps -a
crictl logs --tail=20 <container-id>
Fix
The sequence is: renew, restart the control-plane Pods, refresh copies of admin.conf. If the cluster has several control-plane nodes, the renew command must run on all of them.
1. Renew all certificates
First save the directory that holds the certificates, kubeconfig files and static Pod manifests. The target name must not exist yet, or cp -a would copy the directory inside it. This command should answer "No such file or directory"; if it lists anything, stop and choose another name:
ls -d /root/etc-kubernetes-2026-11-22
Then copy, restrict the copy to root because it holds the CA private key, and prove the copy is complete. The ls -ld line must start with drwx------, and the last ls must list files for both directories; if either check fails, do not continue:
cp -a /etc/kubernetes /root/etc-kubernetes-2026-11-22
chmod 700 /root/etc-kubernetes-2026-11-22
ls -ld /root/etc-kubernetes-2026-11-22
ls /root/etc-kubernetes-2026-11-22/pki /root/etc-kubernetes-2026-11-22/manifests
Disruptive step: the renew command below rewrites the certificates and kubeconfig files in /etc/kubernetes. The reference says renewals "are run unconditionally, regardless of expiration date". The renewed files only take effect once the control-plane Pods are restarted in step 2, so the cluster may be unreachable for a short time.
Run it on each control-plane node (the files it rewrites belong to root, so use a root shell). The renew page says the command uses the existing certificates as the source for names and addresses and does not rely on the kubeadm-config ConfigMap:
kubeadm certs renew all
kubeadm certs check-expiration
To back out, first prove that the saved copy is still complete. Both directories must list files; if either lists nothing or fails, stop, because nothing can be restored from it:
ls /root/etc-kubernetes-2026-11-22/pki /root/etc-kubernetes-2026-11-22/manifests
Then keep a copy of the renewed state. This name must not exist yet either, or cp -a would copy the directory inside it. This command should answer "No such file or directory"; if it lists anything, stop and choose another name:
ls -d /root/etc-kubernetes-renewed-2026-11-22
Then copy the renewed state, and copy the saved files back over the live directory in place, so that /etc/kubernetes is never moved away:
cp -a /etc/kubernetes /root/etc-kubernetes-renewed-2026-11-22
cp -a /root/etc-kubernetes-2026-11-22/. /etc/kubernetes/
kubeadm certs check-expiration
Then restart the Pods again as in step 2. If you have already done step 2, skip the save of the manifests and the mkdir, which already exist. If you have not, do all of step 2 including its save, so that the restored files are the ones in use.
2. Restart the control-plane static Pods
The renew page says the control-plane Pods must be restarted, since dynamic certificate reload is not supported for all components and certificates. Static Pods are managed by the kubelet, so kubectl cannot restart them. The page's method is to remove the manifest from /etc/kubernetes/manifests/ temporarily, wait 20 seconds (the fileCheckFrequency value), and move it back. The manifest names are etcd.yaml, kube-apiserver.yaml, kube-controller-manager.yaml and kube-scheduler.yaml.
First save the manifests. The name /root/manifests-2026-11-22 must not exist yet. This command should answer "No such file or directory"; if it lists anything, stop and choose another name:
ls -d /root/manifests-2026-11-22
Then copy the manifests and make an empty hold directory. After the copy, the last ls must list the four manifest files:
cp -a /etc/kubernetes/manifests /root/manifests-2026-11-22
mkdir /root/manifests-hold
ls /root/manifests-2026-11-22
Disruptive step: moving a manifest out stops that component. With one control-plane node, moving kube-apiserver.yaml or etcd.yaml out makes the cluster API unavailable until the file is moved back.
The page gives no order. This guide restarts one component at a time, etcd.yaml first. Move the manifest out:
mv /etc/kubernetes/manifests/etcd.yaml /root/manifests-hold/
Wait at least 20 seconds, then move it back:
mv /root/manifests-hold/etcd.yaml /etc/kubernetes/manifests/
Wait another 20 seconds, then repeat both commands for kube-apiserver.yaml, kube-controller-manager.yaml and kube-scheduler.yaml in turn.
To back out at any point, first list which manifest is waiting in the hold directory, and prove the saved copy exists:
ls /root/manifests-hold
ls /root/manifests-2026-11-22
Then move the manifest that the first listing shows back, using its own name in place of <manifest-name>:
mv /root/manifests-hold/<manifest-name>.yaml /etc/kubernetes/manifests/
If a manifest in /etc/kubernetes/manifests was damaged, restore that one file from the saved copy, again with its own name:
cp -a /root/manifests-2026-11-22/<manifest-name>.yaml /etc/kubernetes/manifests/
3. Refresh the administrator kubeconfig copy
First check whether $HOME/.kube/config exists. If ls lists it, save it; if ls reports that it does not exist, there is nothing to save, so skip the cp and see the note under the back-out:
ls -l $HOME/.kube/config
Then check that the saved name is free, so that a second run cannot overwrite the saved original. This must answer "No such file or directory"; if it lists a file, stop and choose another name:
ls -l $HOME/.kube/config.2026-11-22
Then save the file:
cp -p $HOME/.kube/config $HOME/.kube/config.2026-11-22
Risk: the two commands below overwrite $HOME/.kube/config, so any other clusters or contexts held in that file are replaced.
The page gives these commands; -i makes cp ask before it overwrites:
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
To undo it when a copy was saved, first check that the saved file exists, then copy it back and test:
ls -l $HOME/.kube/config.2026-11-22
cp -p $HOME/.kube/config.2026-11-22 $HOME/.kube/config
kubectl get nodes
If there was no file before, the undo is to remove the one that the commands created, which is one named path:
rm $HOME/.kube/config
A kubelet client certificate that failed to rotate
The troubleshooting page describes a multi-step repair for a failed node: back up and delete /etc/kubernetes/kubelet.conf and /var/lib/kubelet/pki/kubelet-client* on that node, create a new kubelet.conf with kubeadm kubeconfig user from a working control-plane node that holds ca.key, copy it back and restart the kubelet. It deletes files on a node that is already failing, so no commands are given for it here; it is carried out from that page by a cluster administrator who has saved those files first.
Example (illustrative): a single control-plane node cp01 was built 400 days ago and never upgraded. kubectl get nodes fails. kubeadm certs check-expiration shows apiserver and admin.conf with an EXPIRES date 35 days ago, since 400 − 365 = 35. The engineer saves /etc/kubernetes, runs kubeadm certs renew all, moves each of the four static Pod manifests out and back, then copies the new admin.conf into $HOME/.kube/config. kubectl get nodes answers and check-expiration shows dates one year ahead.
Confirm the fix
kubeadm certs check-expirationshows every EXPIRES date in the future.kubectl get nodesanswers using the refreshed$HOME/.kube/config.kubectl get pods -n kube-systemshows the control-plane PodsRunning.- Every node is
Ready. A node that is not Ready after the renewal points to the kubelet case above.
What to watch afterwards
- Put the earliest EXPIRES date from check-expiration in a calendar, with a reminder well before it.
- Upgrade at least once a year so that kubeadm renews the certificates for you.
- Re-copy
admin.confto every place that holds a copy after any renewal. - On a multi-node control plane, repeat the check on every node, since each holds its own files.
Read next
Related articles you may want next.
- Backup & RecoveryKubernetes etcd Backup and Restore: What It Holds and How to Restore a Control PlaneWhat an etcd snapshot holds and leaves out, when a restore is the wrong fix, and a safe order for restoring a kubeadm control plane.Read article
- HostingExpiring certificates and secrets that silently break hosted applicationsHow to inventory TLS certificates, client credentials and application secrets before they lapse in steady-state hosted production—not only after an outage.Read article
- TroubleshootingKubernetes node NotReady: "Kubelet stopped posting node status"Node NotReady with 'Kubelet stopped posting node status'? Check the kubelet, container runtime, API server path and certificate, then restart safely.Read article
Risks and limits
- These commands have not been run or lab-tested.
- Restoring the saved directory restores the expired certificates too. It returns the node to its earlier state, not to a working one.
- The saved
/etc/kubernetescopy contains the CA key. Store it with the same care as the original. - An etcd snapshot is a further copy worth taking before certificate work; "Kubernetes etcd Backup and Restore: What It Holds and How to Restore a Control Plane" covers what it holds, and this guide gives no snapshot command.
- The Kubernetes pages do not say how the kubelet behaves while
/etc/kubernetes/manifestsor a single manifest is briefly missing, which is why the back-outs copy files back in place instead of moving the live directory away. - The
Unauthorizedformat comes from kubectl source code at release tag v1.34.0 and can differ in other releases. Checkkubeadm certs --helpfor the kubeadm version you run. - The certificates that
kubeadm certs renew allrenews do not include the CAs, and kubeadm does not support rotation or replacement of CA certificates out of the box.
Sources
- Certificate Management with kubeadm — kubernetes.io, last modified 31 October 2026, accessed 22 November 2026
- kubeadm certs — kubernetes.io, last modified 17 August 2024, accessed 22 November 2026
- Troubleshooting kubeadm — kubernetes.io, last modified 9 September 2026, accessed 22 November 2026
- kubeadm implementation details — kubernetes.io, last modified 29 October 2026, accessed 22 November 2026
- Debugging Kubernetes nodes with crictl — kubernetes.io, last modified 3 March 2025, accessed 22 November 2026
- helpers.go (kubectl command utilities) — Kubernetes source code, release tag v1.34.0, no date shown, accessed 22 November 2026
This article was originally published on .