Australian cloud for business software — built and run in Sydney since 2003

+61 2 8282 5656
support@netleverage.com
Talk to an engineer
Talk to an engineer
NetLeverage
NetLeverage

ESXi Host Not Responding or Disconnected in vCenter: Narrow the Cause

Technical article
Troubleshooting

Direct answer: "Not Responding" means vCenter has stopped receiving the host's heartbeats. Broadcom's KB 323612 states that vCenter expects a heartbeat every 10 seconds. If it misses six in a row (60 seconds), it marks the host. The heartbeat travels from the host's vpxa agent to vCenter on UDP 902. The common causes are a blocked or lost UDP 902 path, a wrong managed IP address, a hung hostd or vpxa, or a failed host. "Disconnected" is different. vCenter has stopped managing the host, either because someone disconnected it, a reconnect attempt failed, or the host licence expired. A disconnected host stays disconnected until someone reconnects it (KB 337333). Before restarting anything, prove which part failed. Restarting management agents is a change with documented side effects, not a harmless reflex.

Why it matters

When a host greys out, vCenter-driven operations on it stop. That includes migrations, vCenter-initiated backup sessions, monitoring and changes. The greyed-out state alone does not tell you whether the VMs are still running. Broadcom lists "the host has failed" as one cause of Not Responding. Check the services that run on the host directly before you decide how urgent this is.

vSphere HA's view also changes. KB 337333 states that when a host is Disconnected, vCenter disables HA on it, and its VMs "are therefore not failed over in the event of a host isolation". Leaving a host Disconnected for days is an availability risk, not just a cosmetic one.

How it works

Two states, two meanings

From Broadcom KB 337333:

  • Not Responding: an external factor stops heartbeats arriving. Broadcom names UDP 902 not open, a routing issue, a bad cable, a firewall rule, hostd not running, vpxa not running, or a failed host. The host returns to normal on its own once the cause is fixed.
  • Disconnected: "a state initiated from the vCenter Server side". It happens when a user selects Disconnect, when a Connect on a Not Responding host fails, or when the host licence expires. vCenter stops monitoring the host and "has no knowledge of the health" of it. Someone must reconnect it manually.

The heartbeat path

KB 323612 describes the path:

  • Heartbeats go only from host to vCenter, over UDP 902 by default.
  • vCenter to host uses TCP 902 and 443. A host-to-vCenter TCP 902 test is "expected to fail" and is not a fault.
  • The host learns which address to send heartbeats to from its vpxa configuration. That address has to match the vCenter managed IP in Administration > vCenter Server Settings > Advanced Settings > ManagedIP.

Broadcom names an external or NSX distributed firewall blocking UDP 902 as the most common reason for missed heartbeats. If several hosts flap together, look for a shared cause first: a firewall or network change in that path, or vCenter itself.

Other causes in Broadcom's list

KB 344682 adds causes that make hostd or vpxa unresponsive:

  • packet loss, an unreachable gateway or LACP misconfiguration;
  • switch features such as Duplicate MAC Address Suppression;
  • storage latency or all-paths-down and permanent-device-loss conditions;
  • CPU or memory contention;
  • a vCenter snapshot revert or restore from backup;
  • an MTU mismatch on the management path;
  • a duplicate IP address recorded in vobd.log;
  • an ESXi version newer than the vCenter version.

Read-only checks

None of these was run for this article. SSH to an ESXi host and to the vCenter appliance should be enabled only under your normal change approval, and disabled again afterwards.

  1. Record the state and time. Note whether the host shows Not Responding or Disconnected, and the event text. KB 344682 lists "Cannot synchronize host" in the host's events.

  2. Check whether the VMs' services answer. Test the applications on the host directly, from a client. This decides urgency. It is not a vCenter check.

  3. Check how many hosts are affected. One host points at that host or its uplinks. Many hosts at once points at something they share: vCenter, its network, or a firewall in the heartbeat path.

  4. Read vCenter's log. In /var/log/vmware/vpxd/vpxd.log, KB 344682 shows the state change lines ("connection state changed to [DISCONNECTED]", "marking host as notResponding", "NO_RESPONSE"). The first of these lines is the time vCenter detected the loss, not the time of the last good heartbeat. KB 323612 says the host is marked after six missed 10-second heartbeats, so heartbeats may have stopped up to about a minute earlier. Allow for that when you line the event up with network or host evidence.

  5. Confirm the managed IP and heartbeat port on the host. Per KB 323612, on ESXi 7.0 U3 and later:

configstorecli config current get -c esx -g services -k vpxa_solution_user_config | grep -i server_port configstorecli config current get -c esx -g services -k vpxa_solution_user_config | grep -i server_ip

On ESXi 6.x the values are in /etc/vmware/vpxa/vpxa.cfg (serverPort, serverIp). The port should be 902 unless you deliberately changed it, and the IP should equal vCenter's ManagedIP.

  1. Prove whether heartbeats arrive at vCenter. A UDP test with netcat "will always succeed", so Broadcom recommends capturing on the vCenter appliance instead:

tcpdump -i <vcenter-management-interface> src <host-management-IP> and udp port 902 -nn

Find the interface that carries vCenter's management address first, for example with ip -br addr. A capture on the wrong interface shows nothing and proves nothing. Broadcom's minimum is 10 seconds, because a heartbeat is sent every 10 seconds. To judge cadence, let it run for at least a minute (six intervals), stop it with Ctrl+C and read the "packets dropped by kernel" count tcpdump prints.

Read the result narrowly. Packets arriving about every 10 seconds show those heartbeats reached the appliance during that window. They do not by themselves prove vCenter processed them, or that the path stays clean, so compare the timestamps with the events from step 4. No packets on the right interface points at the host's sender or the network between, once the destination address (step 5) is confirmed correct.

  1. Check the host's own agents and logs. From the DCUI or SSH, KB 344682 points at /var/run/log/vpxa.log ("Can't connect to hostd. Shutting down") and at "hostd detected to be non-responsive" messages on the DCUI. Repeated "Started heartbeating.." lines every minute in vpxa.log match the heartbeat-loss pattern in KB 323612.

  2. Check the management network from the host. From the DCUI, test the default gateway. KB 344682 recommends a capture with pktcap-uw while pinging, to find where packets drop. Also check that the management path MTU matches end to end.

  3. Look for storage or resource pressure. All-paths-down, latency and memory pressure can stall hostd. If the host has crashed, the purple screen article covers evidence capture.

Interpreting the results

Evidence Meaning Next step
State is Disconnected; event shows a user disconnect or licence expiry Administrative state Fix the licence or reconnect; no agent restart needed
Heartbeats arrive about every 10 seconds on the right interface while vCenter still marks the host vCenter-side processing, or loss outside the captured window Extend the capture across a state change; check vCenter health and its logs
No heartbeats in the capture; host's server_ip differs from ManagedIP Host is sending to the wrong address Follow Broadcom's vCenter IP change guidance
No heartbeats; IP and port correct; host gateway ping fails Management network fault Capture with pktcap-uw; involve the network team; check switch features
Network fine; vpxa.log shows it cannot reach hostd Hung agent Consider a targeted agent restart (below)
Agents hung with storage APD/PDL or resource exhaustion Underlying host problem Fix storage or pressure first; an agent restart may not hold
Many hosts flap together Shared cause: a network or firewall change, or vCenter Review recent firewall and network changes; capture for several hosts at once

Decision guidance

Reconnect first, if the cause is fixed. For Not Responding, KB 344682 suggests trying a reconnect after each step. A reconnect has less impact than an agent restart.

Agent restart is a change. Broadcom KB 320280 warns that restarting the management agents may cause temporary disruption, can lead to instability that requires a host reboot, and can affect running tasks and the performance of guest VMs on the host. A hostd restart briefly interrupts communication between vCenter and the host, and Broadcom treats that as expected. Before restarting:

  • check whether a backup job, migration or other task is running against the host, and expect it to fail or need a retry;
  • do not use services.sh restart, or the DCUI option that restarts all management agents, on hosts running vSAN, LACP, NSX or shared graphics for View. Broadcom gives a read-only check for each:

esxcli vsan cluster get esxcli network vswitch dvs vmware lacp status get esxcli software vib list | grep -E 'nsx|vsipfwlib'

On those hosts, restart single services only; - KB 320280's single-service procedure is /etc/init.d/hostd restart followed by /etc/init.d/vpxa restart, or the same from the Host Client under Host > Manage > Services. Allow 1–2 minutes before you judge the result. If a service fails to restart, inspect /var/log/hostd.log.

A hard power cycle is last. KB 344682 places a hard power off and on after a failed agent restart. On a host whose VMs are still running, that stops every VM on it, and HA may restart them elsewhere. Treat it as an outage decision. Where you can, migrate or shut down VMs in an orderly way first.

Rollback: None of the read-only checks needs rollback. If you changed a firewall rule, a managed IP or the heartbeat port, record the previous value so you can restore it.

Verification: After a reconnect or a single-service restart, the host shows as connected in vCenter, recent tasks against it complete, and no new disconnect events appear over the next heartbeat intervals. Compare any changed setting against the value you recorded.

What not to do

  • Do not use a netcat UDP test as proof that heartbeats arrive. Capture on vCenter.
  • Do not remove and re-add the host as a first fix. It is a separate inventory change with its own consequences, and it does not address a blocked heartbeat path.
  • Do not run services.sh restart on vSAN, LACP, NSX or shared-graphics hosts.
  • Do not leave a host in the vCenter-initiated Disconnected state after the fault is fixed. KB 337333 states that disconnecting a host disables HA on it, so its VMs are not failed over in the event of a host isolation until it is reconnected.
  • Do not assume VMs on a greyed-out host are running. Check them.

Limitations

  • No command was run for this article, and no lab validation has been done.
  • Broadcom KB "Updated On" dates were not displayed in the retrieved copies.
  • The effect of an agent restart on any particular backup product's sessions is product-specific and not covered.
  • VCF and SDDC Manager inventory resynchronisation (mentioned in KB 320280) is out of scope.

Acceptance checklist

  • [ ] The state (Not Responding or Disconnected) and the time of the first state change are recorded.
  • [ ] The host's heartbeat target IP and port are checked against vCenter's ManagedIP.
  • [ ] A vCenter-side capture shows whether UDP 902 heartbeats arrive.
  • [ ] The failing layer is named: administrative, network, agent, storage or resources.
  • [ ] Any agent restart was single-service on vSAN, LACP, NSX or shared-graphics hosts, and running tasks were checked first.
  • [ ] The host shows Connected for at least several heartbeat intervals.
  • [ ] HA status for the host is healthy after reconnection.
  • [ ] Interrupted backup jobs or migrations have been re-run and verified.

References

Accessed 2 November 2026. Broadcom "Updated On" dates were not displayed in the retrieved copies.

Related articles

Learning notes

  1. Not Responding means heartbeats stopped. Disconnected is a vCenter-side state that needs a manual reconnect.
  2. Heartbeats go one way, host to vCenter on UDP 902. Prove arrival with a capture on vCenter.
  3. A wrong managed IP on the host gives the same symptom as a firewall block.
  4. An agent restart is a change with documented side effects. Restart only single services on vSAN, LACP, NSX and shared-graphics hosts.
  5. Disconnecting a host from vCenter disables HA on that host, so its VMs are not failed over on host isolation. This is the vCenter-initiated Disconnected state, not heartbeat loss alone. Reconnect it as soon as the cause is fixed.