Start with an end-to-end traffic map, not a new distributed switch. A VMkernel adapter, virtual switch, physical uplink, switch port and destination interface must agree on the path being tested. A successful management ping does not prove that the vMotion or storage path works.
The examples use an isolated three-host lab. Configuration steps are unexecuted; production uplink migration and management cutover require an approved change window and tested console recovery.
1. Map each traffic class
Create a table before touching the network:
| Traffic | Source | Virtual port group | Physical path | Destination |
|---|---|---|---|---|
| Management | Host management VMkernel | PG-MGMT | Approved management uplinks | vCenter, DNS, NTP and operator network |
| vMotion | Dedicated VMkernel | PG-VMOTION | Chosen redundant uplinks | Peer vMotion VMkernel |
| Storage | Storage VMkernel, if used | Storage-specific group | Vendor-supported storage fabric | Storage endpoints |
| Guest application | VM virtual NIC | PG-APP-LAB | Workload uplinks | Application clients and dependencies |
Allocate real lab VLAN IDs, subnets and addresses separately. Record whether each physical switch port is access or trunk, permitted VLANs and expected native VLAN behaviour. Identify the failure domain behind each uplink; two ports on one failed switch are not two independent paths.
Treat VMXNET3 selection as a guest-driver and compatibility decision. Do not replace a working adapter just to change the displayed link speed. A NIC change can affect interface identity and guest IP configuration.
2. Preserve the current configuration
Existing-switch path: if the experiment changes a switch that already exists, export that switch before changing it. Replace VDS-LAB below with its actual lab inventory name and use a unique filename from the management workstation. Do not run this block for a switch that has not been created.
New-switch path: record the existing host, physical-switch and management-path configuration first, then create the separate lab switch in section 3. Export the new switch after its initial configuration is complete and before further experiments. An export of the new switch does not back up the original network.
For an existing distributed switch only:
Get-Help Export-VDSwitch -Full
$vds = Get-VDSwitch -Name 'VDS-LAB' -Server $vc
$dest = Join-Path $PWD ('VDS-LAB-' + (Get-Date -Format 'yyyyMMdd-HHmmss') + '.zip')
Export-VDSwitch -VDSwitch $vds -Destination $dest -Server $vc
Get-VDPortgroup -VDSwitch $vds -Server $vc |
Select-Object Name,PortBinding,NumPorts
Here $vc is an existing authenticated connection to exactly the intended vCenter. An export is a configuration recovery aid, not proof that the management network can survive a bad cutover. Record host uplink assignments and physical switch configuration separately. Export-VDSwitch reference: https://developer.broadcom.com/powercli/latest/vmware.vimautomation.vds/commands/export-vdswitch
3. Create the lab switch and port groups
For a new-switch build, in vSphere Client Networking view create a distributed switch at a version supported by every participating host. For an existing-switch experiment, retain the backed-up switch and change only the approved settings; do not create a duplicate. Assign the planned number of uplinks and give them stable names. Create separate distributed port groups for the traffic classes and set their VLAN policy explicitly.
Use a spare lab host with no production workloads. Broadcom documents the switch preparation and staged uplink workflow: https://knowledge.broadcom.com/external/article/427273/best-practices-for-migrating-esxi-hosts.html; a migration from an existing switch requires its separate maintenance plan. Add one lab host through Add and Manage Hosts. Map a spare physical NIC to the intended distributed uplink while preserving the functioning management path. Add a test VM network first. Verify traffic before moving any VMkernel service.
Create or migrate the dedicated vMotion VMkernel interface with its assigned address and service enabled. Choose its TCP/IP stack intentionally. Moving a VMkernel adapter to a different TCP/IP stack is not a simple in-place stack edit; use a separately planned adapter replacement if that is required. Repeat for the other lab hosts only after the first path is understood. If no spare uplink or independent console path exists, stop at design review; this guide does not make a blind management cutover safe.
Use a teaming policy compatible with the physical switch configuration. Do not select an IP-hash or link-aggregation design on one side and leave the other side as unrelated ports. A distributed-switch diagram is incomplete until the physical-switch operator confirms the matching configuration.
4. Inventory the actual VMkernel path
From an authorised ESXi shell, use documented read-only commands:
esxcli network ip interface list
esxcli network ip interface ipv4 get
esxcli network nic list
Record VMkernel name, address, MTU, switch association and TCP/IP stack. Confirm the intended physical NIC is up at the negotiated speed and inspect error/discard counters for the correct adapter:
esxcli network nic stats get -n vmnic2
vmnic2 is an example, not a recommended interface. Match it to the recorded uplink map. The ESXCLI network reference: https://developer.broadcom.com/xapis/esxcli-command-reference/latest/namespace/esxcli_network.html defines the commands; installed --help takes precedence for a different release.
5. Test connectivity on the correct stack
For a 1500-byte IPv4 path on the default stack, the following example sends three ICMP tests with a 1472-byte payload and fragmentation disabled:
vmkping -I vmk2 192.0.2.22 -s 1472 -d -c 3
The 28-byte difference accounts for ordinary IPv4 and ICMP headers. Do not reuse this calculation as an IPv6 packet-size rule. Replace both the interface and documentation address with allocated lab values.
When the source VMkernel belongs to the dedicated vMotion TCP/IP stack, select that stack:
vmkping -I vmk2 192.0.2.22 -S vmotion -s 1472 -d -c 3
Run the corresponding test in the reverse direction. A default-stack ping can test an entirely different route. VMkernel connectivity guidance: https://knowledge.broadcom.com/external/article/344313/testing-vmkernel-network-connectivity-wi.html
6. Validate jumbo frames only when the design uses them
First establish a working standard-MTU baseline. If the approved design specifies MTU 9000, confirm the virtual switch, VMkernel interfaces and every physical hop support the required frame size. Physical switches may express frame limits differently from an IP MTU; use the switch vendor's definition.
Then test the actual dedicated stack and source interface:
vmkping -I vmk2 192.0.2.22 -S vmotion -d -s 8972 -c 3
A successful small ping followed by a failed large ping narrows the investigation to MTU handling or size-sensitive filtering. It does not identify which switch is wrong. Compare the forward and reverse paths and check every intermediate hop. Broadcom documents the jumbo-frame test: https://knowledge.broadcom.com/external/article/313061/testing-jumbo-frame-pings-over-the-netwo.html.
Do not treat any particular migration-progress percentage as definitive proof of an MTU fault. Use the packet tests and task errors. Do not increase MTU across the environment solely because jumbo frames exist as an option.
7. Distinguish connectivity from performance
Once the path passes connectivity tests, migrate a disposable VM while running its synthetic application transaction. Record migration duration, application response distribution, host CPU, packet drops and the source/destination uplinks used.
Repeat with the same workload and VM memory activity. A memory-intensive VM can produce a different migration workload from an idle VM of the same configured size. A change in migration duration alone is not a network-throughput measurement.
For guest transfers, record payload size, direction, protocol, concurrency and whether endpoints are CPU- or disk-limited. Do not compare a buffered file copy with an unbuffered disk benchmark and attribute the difference to the NIC.
8. Isolate the failed boundary
| Observation | Next check |
|---|---|
| Neither small nor large VMkernel ping works | Addressing, VLAN membership, route, stack, link and ACLs |
| Small works, large fails | MTU at each virtual and physical hop |
| One direction fails | Asymmetric routing, ACLs or inconsistent physical ports |
| VMkernel succeeds but guest traffic fails | Guest VLAN, IP configuration, guest firewall and port-group backing |
| Traffic breaks on one uplink | That uplink's physical VLAN/MTU/teaming configuration |
| Ping succeeds but vMotion fails | Migration compatibility, service binding, task logs and resource conditions |
Acceptance and rollback
Accept the lab design only after both-direction VMkernel tests, guest application checks, repeated migration and an approved uplink-failover exercise pass. A failover exercise must not remove the sole management or storage path. Record which failure was simulated and which paths remained.
If a staged migration fails, keep the proven original management path and restore the recorded port-group/uplink mapping. Switch imports or broad rollback can affect multiple hosts; use the supported client workflow only after reviewing scope. Never delete the original path before validating its replacement.
Related topics: cluster commissioning, vSAN network contention and management-plane security.
Learning notes
- Select the source interface and TCP/IP stack when testing a service network.
- MTU is an end-to-end property.
- A ping proves reachability for that test, not application performance or resilience.
Sources and review status
See the source ledger for dates, version scope and limits, the author self-check, and the series index. Independent review and lab validation remain pending.