Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

How to build and validate a VMware private-cloud cluster

Guide
CloudHostingVMware

Build the cluster in stages: establish host and network consistency, configure resource policies, then prove migration and maintenance behaviour using disposable workloads. A cluster object is not evidence of availability. Its surviving hosts, storage paths and networks must support the workload after a failure.

This guide starts with installed ESXi hosts and a working vCenter appliance. Hypervisor installation, disk formatting, production power-loss tests and storage provisioning are excluded. Procedures are research-based lab instructions; they have not been executed for this article.

1. Define the reference architecture

Use three licensed ESXi 8.0.x hosts, compatible CPU families, redundant physical networking and one existing shared datastore supported by all three hosts. The example names are fictional: esx01, esx02, esx03, DC-LAB and CL-LAB.

Plane Example design Evidence required before clustering
Management Dedicated management VLAN and VMkernel interface vCenter reaches each host; forward/reverse DNS and time agree
Migration Separate vMotion VLAN and VMkernel interface Bidirectional tests using the actual source interfaces
Storage Existing supported shared storage design Same datastore visible and accessible on every host
Workload Consistent VM networks A test VM retains connectivity after migration
Recovery access Independent hardware management Operator can reach each host when vCenter is unavailable

These are design choices, not mandatory VLAN numbers or a universal host count. Do not reuse an address from an example without allocating it in the lab.

Record exact ESXi and vCenter builds, firmware/driver combinations, license entitlements, datastore type and guest support. Resolve compatibility against the vendor matrices before adding hosts. A nested cluster can teach workflow but cannot establish production performance or physical fault tolerance.

2. Calculate capacity after a failure

Create a capacity sheet with per-host usable memory, CPU demand at peak, VM reservations and infrastructure overhead. Calculate the state after losing the largest relevant failure domain. With three identical hosts, losing one leaves two hosts of capacity, not three hosts with a warning.

For example, three hosts with a hypothetical 192 GiB of usable workload memory each leave 384 GiB after one host fails. A 400 GiB workload requirement does not fit even if all three hosts look comfortable during normal operation. This arithmetic is an illustration, not a sizing recommendation. Heterogeneous hosts, reservations, affinities and application requirements make the actual admission-control assessment more complex.

Record the expected behaviour of each tier: restart order, infrastructure dependencies, acceptable outage and minimum service capacity. HA restart is not an application availability guarantee.

3. Establish a controlled PowerCLI session

Use an approved PowerCLI release compatible with the target environment. Inspect installed help before executing examples:

Get-Help Connect-VIServer -Full
Get-Help New-Cluster -Full
Get-Help Set-Cluster -Full
$vc = Connect-VIServer -Server 'vc01.example.invalid'

Use interactive authentication or the organisation's approved session mechanism. Do not disable certificate validation to get past an unexpected certificate. Resolve trust and identity first. Keep this session scoped to the selected vCenter; pass -Server $vc in inventory operations.

Create only an empty lab cluster after confirming the datacentre exists and the name is unused:

$dc = @(Get-Datacenter -Name 'DC-LAB' -Server $vc)
if ($dc.Count -ne 1) { throw 'Expected exactly one lab datacentre' }
if (Get-Cluster -Name 'CL-LAB' -Server $vc -ErrorAction SilentlyContinue) {
    throw 'Cluster already exists; inspect it instead of recreating it'
}
$cluster = New-Cluster -Name 'CL-LAB' -Location $dc[0] `
    -HAEnabled -DrsEnabled -DrsAutomationLevel Manual -Server $vc

This changes inventory. It does not add hosts, provision storage or establish admission capacity. The parameter meanings are documented in New-Cluster: https://developer.broadcom.com/powercli/latest/vmware.vimautomation.core/commands/new-cluster. Inspect a failed creation in vCenter before retrying; an API timeout does not prove that nothing was created.

4. Add hosts and validate placement prerequisites

In the vSphere Client, open the empty cluster and use Add Hosts. Add one host at a time, verify the presented identity, and review discovered networks and datastores before continuing. Do not migrate a production host into this lab design.

For each host, inspect Configure > Networking > VMkernel adapters. Confirm that the intended interface has the vMotion service enabled and that the TCP/IP stack matches the design. Confirm physical uplink assignments, VLANs and MTU across the complete path. Guide 2 provides interface-specific checks.

Check that the shared datastore is the same storage object on each host, not merely another datastore with a similar display name. Check that the VM port-group backing exists everywhere a workload may run. Stop if either check fails; DRS cannot repair an incomplete network or storage design.

5. Configure HA, DRS and CPU compatibility deliberately

Open cluster Configure > vSphere Availability. Enable host monitoring and admission control. Select the policy appropriate to the documented failure tolerance and examine the calculated reserve. Do not disable admission control simply because a test VM cannot start: that rejection may expose insufficient failover capacity.

Under vSphere DRS, retain Manual automation during commissioning. Review recommendations and resolve incompatible affinity rules before permitting automatic migrations. Broadcom distinguishes Manual, Partially Automated and Fully Automated behaviour; these are separate operational choices. Cluster creation guidance: https://knowledge.broadcom.com/external/article/308413/creating-a-vsphere-drs-and-ha-cluster-us.html

Choose an EVC baseline only after checking every host's CPU support and the guest requirements. EVC is not a bridge between arbitrary CPU vendors. Existing powered-on VMs may constrain a baseline change; resolve that through a planned lifecycle operation, not forced power actions.

After acceptance, changing DRS automation is explicit and reversible:

$cluster = Get-Cluster -Name 'CL-LAB' -Server $vc
Set-Cluster -Cluster $cluster -DrsAutomationLevel FullyAutomated -Server $vc

Set-Cluster: https://developer.broadcom.com/powercli/latest/vmware.vimautomation.core/commands/set-cluster is a configuration command. Run it only after the lab migration tests pass. Restore the recorded previous level if automatic movement is not acceptable.

6. Prove migration and planned maintenance

Deploy two disposable VMs on the shared datastore. Give them synthetic data and a small repeatable transaction, such as creating and retrieving a test record through a lab service.

  1. Run the transaction repeatedly and record timestamps and failures.
  2. Migrate a powered-on test VM to each other host using Change compute resource only. Read the compatibility report before submitting.
  3. Confirm the destination host, application response, datastore accessibility and network identity after every move.
  4. Request maintenance mode for one lab host through the normal evacuation workflow. Confirm the remaining hosts can accommodate the VMs and maintain service.
  5. Exit maintenance mode, confirm host health and review DRS recommendations.

A successful maintenance evacuation proves a planned path. It does not prove HA recovery after abrupt host loss. That requires a separately approved, isolated fault-injection exercise with an application acceptance test; this unvalidated guide does not provide a production host power-off recipe.

7. Troubleshoot by the failed boundary

Symptom Check first Avoid
Migration compatibility fails CPU baseline, devices, networks and shared storage Repeatedly retrying the same rejected task
Migration starts then stalls VMkernel path, MTU, packet loss and source/destination load Increasing every timeout
VM cannot power on under admission control Reservation and failover-capacity calculation Disabling admission control to hide the constraint
HA agent unhealthy Management connectivity, DNS/time and host events Assuming vCenter reconnect equals HA acceptance
DRS makes no recommendation Rule constraints, automation setting and resource imbalance Treating equal VM counts as balanced demand

Acceptance, rollback and limits

Retain the inventory export, policy settings, VM migration events, transaction results and measured maintenance impact. Acceptance requires healthy hosts, accessible shared storage, functioning VM networks and successful transactions after every tested move.

Restore recorded DRS/HA policies through the client if a policy experiment fails. A workload migration is reversed through a new checked migration, not by restoring an old management-plane snapshot. Do not remove a host or cluster while it owns test resources. Cleanup is a separate reviewed operation.

This guide does not validate multi-site quorum, vSAN availability, stretched clusters or application clustering. Related guides cover VMware networking, resource contention and recovery exercises.

Learning notes

  • HA capacity must exist after the failure, not only before it.
  • DRS placement, vMotion success and application availability are different tests.
  • Manual commissioning makes policy decisions visible before automation takes over.

Sources and review status

See the source ledger for dates, version scope and limits, the author self-check, and the series index. Independent review and lab validation remain pending.