Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

SFTP Host Key Changed: Verify It and Rotate Keys Safely

Technical article
Cloud

Direct answer: A changed host key means the server presented a different key from the one your client recorded for that name. That is either a legitimate change (the server was rebuilt, restored, moved or had its keys rotated) or a sign that something is sitting between you and the real server. You cannot tell which from the client side alone. So:

  1. Stop the job. Do not let it retry with checking turned off.
  2. Get the server's real fingerprint through a channel the attacker cannot control. Ask the server's owner, read it from their console, or use a DNSSEC-validated SSHFP record.
  3. Compare. If it matches, replace only that host's entry in the job's known_hosts. If it does not match, treat it as a security incident.

For planned changes, avoid the break entirely. Publish the new host key alongside the old one, let clients learn it (OpenSSH's UpdateHostKeys does this automatically in many setups), and only then remove the old key. Rotate the transfer account's own key the same way: add the new key, switch the job, then remove the old one. Never set StrictHostKeyChecking=no as the fix.

Why it matters

The SSH protocol rests on the client knowing the server's public key in advance. RFC 4251 states that the host key is used during key exchange "to verify that the client is really talking to the correct server", and that this requires prior knowledge of the server's key. When that knowledge is missing or ignored, the encryption still works, but it may be encryption to the wrong party. RFC 4255 puts it plainly: accepting a key without verifying the fingerprint through a secured channel leaves the connection vulnerable to a man-in-the-middle attack.

For an automated transfer this matters more than for an interactive login:

  • The data is the target. Backup archives, exports and payroll files are exactly what an interceptor wants. A transfer to an impostor delivers them.
  • Credentials can leak. With password authentication, a client that connects to an impostor hands it the password. With key authentication the private key is not disclosed, but the impostor still receives the uploaded data and can return what it likes on download.
  • Nobody is watching. A job that has been "fixed" by disabling host checking keeps running quietly for years and will accept any future impostor too.

A changed key is also a common operational symptom:

  • Rebuild or restore: the server was rebuilt from scratch, or restored to a state with different keys.
  • Clone: a VM was cloned from a template that carried the template's keys (see the duplicate-identity article listed under Related articles).
  • Name moved: a DNS name or address was moved to a new server.
  • Load balancing: several back-end servers behind one name each present their own key.
  • Provider rotation: the provider rotated keys without telling customers.

Each needs a different fix. None of them is fixed by turning checking off.

How it works

What the client stores

An OpenSSH client keeps the keys it trusts in known hosts files. By default these are the user's ~/.ssh/known_hosts (and known_hosts2), plus the system-wide /etc/ssh/ssh_known_hosts (and ssh_known_hosts2). The UserKnownHostsFile and GlobalKnownHostsFile options change those paths. The sshd(8) manual gives the line format: an optional marker, host name patterns, key type, the base64 key, and an optional comment that "is not used".

Points to know:

  • Ports: a server on a non-standard port is stored as [host]:port.
  • Hashed names: names may be hashed (lines starting with |), which hides them from anyone reading the file. You then need ssh-keygen -F to find a host's line.
  • Markers: @revoked marks a key that must never be accepted. @cert-authority marks a certificate authority key that vouches for host certificates.
  • Matching: authentication succeeds if any matching line has the right key. That is what makes an overlap rotation possible: two keys for one host can both be trusted at once.
  • Aliases: HostKeyAlias makes the client look up and save the key under an alias rather than the real host name. That is useful when several names reach one server, or one name reaches several.

What the client does when the key is unknown or different

StrictHostKeyChecking controls this. The ssh_config(5) manual describes the values:

Value New host Changed key
yes Never added automatically; you must add it yourself Refuses to connect
accept-new Added automatically Refuses to connect
ask (the default) Added only after you confirm Refuses to connect
no or off Added automatically Connection allowed to proceed, "subject to some restrictions"

BatchMode yes disables interactive prompts, including host key confirmation. That is the usual setting for scheduled jobs. For an automated job, the safe combination is a known hosts file you populated after verification, plus StrictHostKeyChecking yes. Then an unknown or changed key stops the job instead of being trusted silently. accept-new protects against changes but trusts the very first connection blindly. RFC 4251 notes that skipping the check on first connection leaves you open to an active man-in-the-middle and should not normally be allowed by default.

How a server can introduce a new key without breaking clients

A server may hold several host keys at once. sshd_config(5) says "it is possible to have multiple host key files", and RFC 4251 allows multiple host keys using different algorithms. OpenSSH servers from 6.8 onward support a protocol extension (hostkeys@openssh.com) that tells the client about all of the server's host keys after authentication.

The client option UpdateHostKeys decides whether to accept those notifications and add the extra keys to the user known hosts file. The manual is precise about the conditions:

  • What is accepted: additional keys are accepted only if the key that authenticated the host was already trusted, or was explicitly accepted by the user.
  • Which file: the host must have been authenticated through UserKnownHostsFile, not GlobalKnownHostsFile.
  • Plain keys only: the host must have been authenticated with a plain key, not a certificate.
  • Default: it is enabled by default "if the user has not overridden the default UserKnownHostsFile setting and has not enabled VerifyHostKeyDNS"; otherwise it is no. The OpenSSH 8.5 release notes (3 March 2021) record the change that turned it on by default.

The manual describes the purpose as supporting "graceful key rotation by allowing a server to send replacement public keys before old ones are removed".

The practical consequence for automated jobs is that many of them do set a custom UserKnownHostsFile (one per job or service account). For those, UpdateHostKeys is off unless you set it explicitly. Such jobs do not learn new keys on their own, and a rotation will break them unless you update their file.

Other ways to establish trust

  • SSHFP records in DNS (RFC 4255). The server's fingerprint is published as a DNS record. RFC 4255 says a key verified this way "MUST NOT be trusted" unless the record was authenticated by DNSSEC. A client that does not validate DNSSEC itself must use a secure transport to the resolver that does. In OpenSSH, VerifyHostKeyDNS yes trusts keys that match a secure fingerprint from DNS and treats insecure ones as ask. Note that enabling it turns UpdateHostKeys off by default.
  • Host certificates. The client trusts a certificate authority key (an @cert-authority line), and the server presents a host certificate signed by it. RFC 4251 describes this as the second trust model. It eases rotation at the cost of running a CA. This article does not cover setting one up.

What the server's administrator can see

The fingerprint of a host key comes from the public key file on the server. The ssh(1) manual gives the example ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub. OpenSSH shows SHA-256 fingerprints by default (FingerprintHash). If someone gives you an older MD5 fingerprint, ssh-keygen -E md5 produces the matching format. A fingerprint read on the server's own console is the strongest out-of-band reference you can get.

Read-only checks

None of these was run for this article. Unless noted, these checks read local files or print configuration and change nothing. Run them as the account the transfer job runs as, because that account's configuration and known hosts file are the ones that matter. Do not paste private keys, passwords or session output containing credentials into tickets. Fingerprints and public keys are not secrets.

1. Which files and options does the job actually use?

# Prints the evaluated client configuration and exits without opening an SSH
# session. A 'Match exec' block still runs its command (see below).
ssh -G sftp.example.net | grep -Ei '^(hostname|port|user|userknownhostsfile|globalknownhostsfile|stricthostkeychecking|updatehostkeys|hostkeyalias|verifyhostkeydns|batchmode|identityfile) '

ssh -G evaluates Host and Match blocks and prints the result without establishing a session to the server. It is not guaranteed to be free of side effects: ssh_config(5) documents Match exec, which runs a shell command while the configuration is evaluated, and that command can do anything, including network access. Before running it, look for such blocks in the files the job uses, for example grep -n -i 'match.*exec' ~/.ssh/config /etc/ssh/ssh_config /etc/ssh/ssh_config.d/*, and read what each one runs. If the job passes options on its command line (sftp -o ..., -F configfile, -i keyfile), include them, because they override the files. Record the known hosts file paths. Every later check refers to them.

2. What does the job currently trust for that host?

# Read-only. Lists known_hosts entries for the host (and port, if non-standard).
ssh-keygen -F sftp.example.net -f /path/to/job/known_hosts
ssh-keygen -F '[sftp.example.net]:2222' -f /path/to/job/known_hosts
# Read-only. Prints fingerprints of every key in the file.
ssh-keygen -l -f /path/to/job/known_hosts

-F finds the entries even when the file uses hashed names. Note the key types and fingerprints recorded for the host. Look as well for entries under the server's IP address (CheckHostIP adds them when enabled) and under any HostKeyAlias.

3. What is the server presenting now?

# Contacts the server's SSH port without logging in. Output is NOT verified.
ssh-keyscan -p 22 -t ed25519,ecdsa,rsa sftp.example.net > /tmp/presented-keys.txt
ssh-keygen -l -f /tmp/presented-keys.txt

The ssh-keyscan(1) manual says it "cannot verify the authenticity of the host keys it obtains". An attacker on the path could substitute their own key, so its output must be verified out of band. Use this check to see what is being presented, not to decide whether to trust it. It makes one connection per requested key type and prints the key negotiated on each, so it does not list every key the server has configured; two keys of the same type cannot both appear. It is network traffic to the server, so make sure you are allowed to scan it. Alternatively, a failing job's verbose log (sftp -v) shows the presented key's fingerprint.

4. Get the reference fingerprint from a separate channel

Ask the server's owner for the SHA-256 fingerprints of the server's host public keys. Use a channel that does not go through the network path you are worried about: a phone call to a known contact, a signed ticket in the provider's portal, or their published documentation over HTTPS. Even better, ask them to read the fingerprints from the server's console with ssh-keygen -l -f on each /etc/ssh/ssh_host_*_key.pub. If you administer the server yourself, read them on its console, not over the SSH connection whose trust is in question.

If the provider publishes SSHFP records and your resolver path validates DNSSEC, the following query is a further independent source:

# Read-only DNS query. Trust it only if DNSSEC was validated locally, or by a
# validating resolver reached over a secure path; the 'ad' flag alone over an
# untrusted path is not enough.
dig +dnssec SSHFP sftp.example.net

5. Is something else answering on that name?

Compare what the name resolves to now with what it resolved to when the job last worked: check the job's previous logs, your DNS change records and the provider's notices. A changed address together with a changed key fits a migration. Different keys on successive connections fit a load-balanced pool whose members have different keys. An unchanged address with a changed key and no notice from the owner is the case that needs the most suspicion.

6. On the server (if you administer it): which keys are configured?

# Read-only on the server. Shows the effective sshd configuration, including HostKey lines; needs root.
sshd -T | grep -Ei '^(hostkey|hostcertificate|hostkeyalgorithms) '

sshd -T performs a configuration test and prints the effective settings without restarting anything. Compare the listed host keys' fingerprints with what clients see.

Interpreting the results

What you find Likely meaning Next step
Presented fingerprint matches the owner's out-of-band fingerprint Legitimate change (rebuild, migration, rotation) Decision 1: replace the entry deliberately
Presented fingerprint does not match the owner's fingerprint Possible interception, or you have the wrong server or port Decision 2: incident handling; do not connect
Owner cannot supply a fingerprint Trust cannot be established Keep the job stopped; escalate for a verified fingerprint; do not "accept for now"
Different fingerprints on repeated scans, each matching one of the owner's servers Load-balanced pool with per-server keys Ask the owner to use one key set across the pool, or trust each published key deliberately (Decision 1)
Known hosts has only an old key type; server now offers a newer type as well Algorithm change on the server Verify the new key and add it; keep the old one until the owner retires it
Job uses a custom UserKnownHostsFile and broke after a planned rotation UpdateHostKeys was off for that job Decision 1 now; Decision 3 for next time
Server is a recently cloned or restored VM It may carry another server's keys, or have regenerated new ones Confirm with its owner which keys are intended; see the duplicate-identity article
Valid SSHFP record with DNSSEC matches presented key Independent confirmation Proceed with Decision 1, recording the DNSSEC evidence

Decision guidance

Make changes under change control, and copy the job's known hosts file before editing it. A known hosts file is not secret, but it is the job's whole trust store.

Decision 1: Replace a verified changed key

Only after Check 4 confirms the new fingerprint:

# Changes the job's known_hosts. Take a copy first.
cp -p /path/to/job/known_hosts /path/to/job/known_hosts.before-change

# Read-only first. Use the identity the job really looks up (Check 1: port,
# HostKeyAlias, CheckHostIP). Run ONE of these, for the affected endpoint:
ssh-keygen -F sftp.example.net -f /path/to/job/known_hosts               # port 22
ssh-keygen -F '[sftp.example.net]:2222' -f /path/to/job/known_hosts      # port 2222
grep -n '^@' /path/to/job/known_hosts      # CA and revocation marker lines

# Only if each matching line names this host alone: remove those lines.
ssh-keygen -R sftp.example.net -f /path/to/job/known_hosts               # or the [host]:port form

ssh-keygen -F prints the line number of each match. A known hosts line can list several comma-separated names, addresses or patterns, and ssh-keygen -R removes a matching line in its entirety, so removing this host can also remove the trust of another host on the same line. Marker lines (@cert-authority, @revoked) are kept by -R; leave them alone, because they carry the CA and revocation decisions. Where a matching line is shared, edit it by hand instead: delete only this host's name from it, then add the new line for this host. Run only the command for the endpoint that changed; the default-port and [host]:port entries are different trusts.

Then add the verified key. The safest source is the public key line supplied by the owner over the trusted channel. Failing that, use the line from Check 3 whose fingerprint you have just matched against the owner's fingerprint. Append that line, and only that line, to the job's known hosts file. If the owner has published several keys (for example Ed25519 and RSA), add each verified key. Remove entries under the IP address or alias as well if they hold the old key.

Do not remove host entries from shared or system-wide known hosts files to fix one job. Edit the file the job uses, as shown by Check 1.

  • Verification: diff /path/to/job/known_hosts.before-change /path/to/job/known_hosts shows only the lines you meant to change, and other hosts' lines and marker lines are untouched. ssh-keygen -F shows only the new verified key or keys for the host. A manual run of the job as its own account, with StrictHostKeyChecking yes, connects without a prompt, and -v output shows the matching fingerprint. The next scheduled run succeeds. A manual run alone does not prove that.
  • Rollback: Reverse only your own edit, using the copy as the reference: remove the line you added and put back the line you removed. Do not copy the whole file back, because other entries may have changed since (another job, or UpdateHostKeys). Restoring the old trust will fail against a server that has really changed, so rollback here mainly means "stop and investigate".

Decision 2: The fingerprint does not match

Treat it as a possible interception:

  1. Keep the job stopped. Do not connect manually "just to look". With password authentication, a connection would hand over the password.
  2. Record the evidence: the presented fingerprint, time, resolved address, source network and job log.
  3. Tell the server's owner and your security contact through the trusted channel.
  4. Check the path: DNS answers from different resolvers, proxies or NAT devices that could redirect traffic, and recent network changes.
  5. Rotate whatever may have been exposed. If the job used password authentication and connected to the unverified server at any point, treat the password as exposed and change it through the owner's normal process. For key authentication the private key was not disclosed, but treat any data uploaded during that window as delivered to an unknown party.
  • Verification: The owner confirms the correct fingerprint, the path issue is identified and fixed, and the presented key then matches. Proceed with Decision 1.
  • Rollback: None is needed. Nothing has been changed.

Decision 3: Rotate a server's host key without breaking clients (server owners)

  1. Generate the new key on the server. Keep the private key readable by root only; sshd refuses group- or world-readable key files. Add a HostKey line for it in addition to the existing ones.
  2. Test and reload. Run sshd -t to check the configuration, then reload sshd in the way your platform does it. The sshd(8) manual says sshd rereads its configuration when it receives SIGHUP. Confirm on your platform how its service manager reloads sshd and whether established sessions are affected.
  3. Publish the new fingerprint to every client owner through your trusted channel, with a date by which the old key will be removed. Update SSHFP records if you publish them.
  4. Let clients pick up the new key. Clients with UpdateHostKeys in effect and an already-trusted old key learn it automatically after their next authentication. Clients with custom known hosts files, non-OpenSSH clients and appliances must add it themselves after verifying it.
  5. Remove the old key after the overlap period. Delete its HostKey line, test, reload, and keep the old key material according to your retention policy rather than leaving it on the server. Clients will drop the old key from their files when UpdateHostKeys sees it gone. Others should remove only the old key's entry and add the new key, as in Decision 1, taking care with shared lines.
  • Verification:
  • During the overlap, the server administrator confirms on the console, with ssh-keygen -l -f on each configured HostKey file and sshd -T, that both keys are configured.
  • A test client with UpdateHostKeys in effect and the old key trusted authenticates once, and ssh-keygen -F on its known hosts file then shows the new key; clients provisioned by hand have the verified line added.
  • Before the old key is removed, a fresh connection from a test client that trusts only the new key (StrictHostKeyChecking yes) authenticates without a prompt.
  • ssh-keyscan shows only the negotiated key for each type it requests, so it is not proof that both keys are configured.
  • Owners of the key clients confirm they have the new fingerprint, and after the old key is removed every client's next scheduled run succeeds.
  • Rollback: During the overlap, removing the new HostKey line and reloading puts the server back as it was. After removal, restore the old key's HostKey line from your retained material, test and reload.

Decision 4: Rotate the transfer account's own key (client side)

The client key authenticates the job to the server. Rotate it with an overlap too:

  1. Generate a new key pair on the job host, stored with the same protection as the current one. Do not copy private keys between hosts or into tickets.
  2. Add the new public key to the server account's authorized keys alongside the old one. With managed SFTP services, use the provider's key management interface instead of editing files. The sshd(8) manual documents options such as from="pattern-list" and restrict. Use them to limit where the key may be used from and what it may do. expiry-time="timespec" lets a key stop being accepted after a date, which makes forgotten keys expire on their own.
  3. Switch the job to the new key (IdentityFile, or -i on its command line), and run it once manually.
  4. Remove the old public key from the server after the next scheduled run succeeds. Then remove the old private key from the job host.

For removing the keys of people who have left, rather than rotating a service key, see the access revocation article listed under Related articles.

  • Verification: Server logs show the job authenticating with the new key's fingerprint. The next scheduled run succeeds. After removal, an attempt with the old key is refused.
  • Rollback: Until step 4, switch the job back to the old key. After step 4, re-add the old public key only if you still trust its private key.

What not to do

  • Do not set StrictHostKeyChecking no or off, or point UserKnownHostsFile at /dev/null, to make a failing job run. The first trusts any server; the second forgets every server.
  • Do not trust ssh-keyscan output, or the first connection under accept-new, as verification. Both just record what was presented.
  • Do not delete a whole known hosts file to clear one entry. Change only the affected entry in the job's own file (Decision 1).
  • Do not remove the old host key before clients have the new one. That turns a planned rotation into an outage.
  • Do not put passwords or private keys into scripts, tickets or chat to get a job working.
  • Do not leave a cloned server with the template's host keys if its owner intends it to be a separate server. Two servers with one identity make host key checking meaningless between them.

Limitations

  • OpenSSH only: The behaviour described is OpenSSH's, from the current OpenBSD manual pages. Packaged versions (Linux distributions, Windows OpenSSH) can lag behind or patch defaults. Check ssh -V and your platform's manual.
  • Other clients and servers: Other SFTP clients (graphical tools, libraries, appliances) and managed SFTP services store and rotate keys their own way. Use their documentation for the equivalent steps.
  • SSHFP: This is only as trustworthy as the DNSSEC validation behind it. The article does not cover DNSSEC deployment.
  • Host certificates: These are mentioned but not covered.
  • No lab testing: None of the commands was run for this article.

Acceptance checklist

  • [ ] The job's effective known hosts file and options are recorded (Check 1).
  • [ ] The new fingerprint was matched against one obtained through a separate trusted channel (Check 4), and the channel is recorded.
  • [ ] Only the affected host's entries were changed in the job's own file, shared lines and marker lines preserved, a copy kept and the diff reviewed (Decision 1).
  • [ ] The job runs with StrictHostKeyChecking yes (or ask with BatchMode yes), never no.
  • [ ] The next scheduled run succeeded.
  • [ ] For server rotations: both keys were published for an overlap period, client owners were notified, and the old key was removed only afterwards (Decision 3).
  • [ ] For client key rotations: the old public key was removed from the server after the new key worked (Decision 4).
  • [ ] Any mismatch was handled as a security event, and exposed passwords were changed (Decision 2).

References

All accessed 2 November 2026.

  • IETF, RFC 4251, "The Secure Shell (SSH) Protocol Architecture", section 4.1 Host Keys: role of the host key, multiple host keys, local database and CA trust models, first-connection risk. https://datatracker.ietf.org/doc/html/rfc4251
  • IETF, RFC 4253, "The Secure Shell (SSH) Transport Layer Protocol": server host key authentication during key exchange. https://datatracker.ietf.org/doc/html/rfc4253
  • IETF, RFC 4255, "Using DNS to Securely Publish Secure Shell (SSH) Key Fingerprints": SSHFP record, DNSSEC requirement, man-in-the-middle risk of unverified acceptance, rotation and revocation through DNS. https://datatracker.ietf.org/doc/html/rfc4255
  • OpenBSD manual, ssh_config(5): StrictHostKeyChecking values, UpdateHostKeys conditions and default, VerifyHostKeyDNS, CheckHostIP, HostKeyAlias, HashKnownHosts, BatchMode, known hosts file defaults, FingerprintHash. https://man.openbsd.org/ssh_config.5
  • OpenBSD manual, ssh(1): verifying host keys, server-side fingerprint example, -E for MD5 comparison, SSHFP example, -G. https://man.openbsd.org/ssh.1
  • OpenBSD manual, ssh-keygen(1): -l, -F, -R, -H, -r, -E. https://man.openbsd.org/ssh-keygen.1
  • OpenBSD manual, ssh-keyscan(1): purpose, cannot verify authenticity, -D. https://man.openbsd.org/ssh-keyscan.1
  • OpenBSD manual, sshd(8): known hosts file format and markers, authorized_keys options (expiry-time, from, restrict), -t, -T, SIGHUP reload. https://man.openbsd.org/sshd.8
  • OpenBSD manual, sshd_config(5): HostKey defaults, multiple host keys, key file permissions, HostCertificate. https://man.openbsd.org/sshd_config.5
  • OpenBSD manual, sftp(1): uses the ssh transport and accepts ssh options. https://man.openbsd.org/sftp.1
  • OpenSSH release notes: UpdateHostKeys enabled by default in OpenSSH 8.5 (2021-03-03); hostkeys notification extension. https://www.openssh.com/releasenotes.html

Related articles

Learning notes

  1. A changed host key cannot be judged from the client. Compare it with a fingerprint obtained through a separate trusted channel before trusting it.
  2. StrictHostKeyChecking yes with a verified, job-specific known hosts file is the safe setting for automation. no trusts anyone; accept-new trusts the first answer.
  3. ssh-keyscan and first-connection acceptance only record what was presented, one negotiated key per type. They are not verification.
  4. Servers may hold several host keys at once. Rotate by adding the new key, letting clients learn it (UpdateHostKeys, on by default since OpenSSH 8.5 unless a custom known hosts file or VerifyHostKeyDNS turns it off) or provisioning it, proving a fresh authentication with the new key, then removing the old one.
  5. Rotate the job's own key with an overlap too: add, switch, confirm the next scheduled run, then remove the old key.