In a mixed-version cluster managed by vCenter 8.0 U3, DRS can become temporarily unavailable when the only host capable of running Embedded vCLS becomes unavailable. vCenter may need to fall back to External vCLS, and that transition is not instantaneous. Use the cluster's vCLS events and host timeline before treating disappearing service VMs as accidental deletion. A29-S1
This is a source-only diagnostic for the transition between pre-8.0-U3 ESXi hosts and ESXi 8.0 U3 hosts. No upgrade, maintenance operation, vCLS deletion or DRS test was performed.
Why one upgraded host changes the cluster
vSphere Cluster Services (vCLS) supplies services used by DRS. In the documented vCenter 8.0 U3 behaviour, available ESXi hosts running 8.0 U3 or later support Embedded vCLS. Older hosts use the original External vCLS implementation. A29-S1
During a rolling upgrade, the first supported host can start Embedded vCLS. vCenter waits for that service to become available before removing the previous External vCLS VMs. Seeing those external VMs disappear can therefore be consistent with the expected transition.
The reverse direction has a different constraint. If all supported hosts become unavailable, vCenter cannot wait indefinitely for External vCLS to deploy before allowing the host event to occur. A period with neither implementation running can leave DRS unavailable.
“Unavailable” includes more than a failed server. The source lists maintenance, standby, disconnected or not-responding states, and removal from the cluster or inventory. The precise host event matters to the diagnosis.
Collect the two timelines
Prerequisites are the exact vCenter and host builds, existing cluster events and the availability history of every Embedded-capable host. Use read access to existing evidence. Do not move a host into maintenance mode to reproduce the condition.
Create one timeline for host state and another for cluster-service transitions. Broadcom identifies these event types:
com.vmware.vc.vcls.TransitionedToEmbeddedEvent
com.vmware.vc.vcls.TransitionedToExternalEvent
com.vmware.vc.vcls.DegradedEvent
com.vmware.vc.vcls.NonHealthyEvent
The first two describe implementation transitions. The latter two describe unhealthy service state. Retain the timestamp and cluster identity for each; a similarly named event from another cluster is not evidence for this outage. A29-S1
Record whether the supported host was actually available, rather than merely present in inventory. An upgraded host still in maintenance mode does not provide the same service opportunity as an available one.
Work through a mixed-cluster example
Consider a synthetic four-host cluster. Three hosts remain on an earlier 8.0 update. Host H4 has reached 8.0 U3 and leaves maintenance mode. The cluster records a transition to Embedded vCLS and the external service VMs disappear.
Later H4 returns to maintenance mode. No other available host supports Embedded vCLS. The cluster records unhealthy vCLS and attempts a transition to External vCLS; DRS is unavailable during the gap.
This sequence supports the documented fallback explanation. It does not prove that application VMs have stopped, that every cluster service failed or that the virtual network is broken. Record workload symptoms separately instead of promoting a DRS status into a general outage claim.
If two supported hosts remained available throughout the incident, the simple “last capable host disappeared” explanation does not fit. Preserve that counter-evidence and investigate the actual service failure rather than forcing the incident into this example.
Distinguish a transition from a stuck recovery
Broadcom describes the common case as temporary. It also describes a worse case where the remaining cluster cannot support External vCLS, including insufficient resources for deployment. That is why waiting for an arbitrary fixed number of minutes is not an evidence-based acceptance criterion. A29-S1
Read existing deployment errors and resource-availability evidence if the unhealthy state persists. Do not delete vCLS VMs, enter Retreat Mode or change resource settings merely to make the inventory look familiar. Those operations alter the control service and need a separately reviewed plan.
For future maintenance planning, the source explains why having more than one available Embedded-capable host reduces dependence on a single host. Apply that observation to the approved upgrade design; it is not permission to accelerate an unvalidated upgrade.
KB 370351 gives two recovery decisions: provide the resources needed for External vCLS, or make the Embedded-capable host available again so the cluster re-upgrades and deploys a new Embedded vCLS VM. Neither is an action performed here. Read-only inventory can also show the expected downgrade pattern: no Embedded vCLS VMs, possibly External vCLS VMs, and Embedded-capable hosts present but unavailable. Match this with the events; inventory alone does not establish the cause.
Acceptance and limits
A complete diagnostic aligns the supported-host availability change, implementation transition and DRS health interval. If the sequence is incomplete, report the missing event or host evidence explicitly.
After an independently approved recovery, proposed acceptance includes healthy vCLS and restored DRS availability, with no unresolved deployment failure. A later migration or placement test is a separate authorised activity. No such result is claimed here.
Inspection needs no rollback. A maintenance or service-mode change requires its own validated reversal plan; recreating deleted service VMs is not an appropriate diagnostic technique.
For upgrade planners, the KB documents Retreat Mode before a rolling upgrade as a proactive mitigation, explicitly suspending DRS until enough hosts have completed the upgrade. It also describes keeping the first upgraded host in maintenance mode until a second is upgraded, then returning both together. These are separately reviewed upgrade strategies with cluster-wide consequences, not diagnostic instructions; the warning above concerns using them impulsively during an incident.
Learning notes
- External vCLS VMs disappearing can reflect an expected upgrade transition.
- The last available Embedded-capable host creates a temporary dependency.
- DRS health, vCLS implementation and application availability are separate observations.