Backup slow or "Cannot use mode hotadd": which VDDK transport mode ran

A backup finished but moved data slowly, or the job log says one transport mode could not be used and another was tried, or backups hang while the host logs complain about memory for "NFC". All three trace back to how the Virtual Disk Development Kit (VDDK) carried the disk data to the backup proxy.

VDDK is VMware's library that backup products use to read virtual disks. It offers four transport modes: NBD, NBDSSL, HotAdd and SAN. When a mode is not possible, a job can move to another without failing, unless your backup product sets its own list. A job can therefore run on a slower mode than planned, with only a log line to show it.

This article explains how each mode moves data, what each needs, the log lines that show a rejection, and how to choose. The proof steps only read. One gap: no official page with SAN failure messages could be fetched, so SAN rejections have no quoted log text here.

What is actually happening

The proxy is the machine that runs your backup product's VDDK code, physical or virtual.

NBD / NBDSSL   datastore -> ESXi host (NFC service) -> network path to the proxy -> proxy
HotAdd         datastore -> disk attached to the proxy VM -> proxy reads a local disk
SAN            datastore LUN -> storage fabric -> proxy reads the LUN directly

Several requirements below come from pages written for vSphere 5 and 6; no release notes could be fetched to show whether later releases relaxed them.

NBD stands for network block device. The ESXi host reads the disk and sends the data over the network to the proxy, using the host's Network File Copy (NFC) service. NBD is built into the library, so it is always available given a network path to the host, but large disks can take a long time. As of vSphere 7.0, ESXi hosts support a dedicated network for NBD traffic: the Broadcom NBD best-practices page describes the vSphereBackupNFC service tag. Changing that network is a separately reviewed configuration operation, not a read-only diagnostic step.

NBDSSL is the same path with the data encrypted. The documentation says NBD is faster and lighter than NBDSSL, and VMware recommends encryption for sensitive data even on a private network.

HotAdd attaches the disks of a snapshot or linked clone of the protected machine to a proxy that is itself a virtual machine, so the proxy reads them as local disks. The documentation says the proxy must have access to the same datastore as the target, and that the VMFS version and data block sizes of the target must match the datastore where the proxy resides. It works only on machines with SCSI disks (IDE is not supported) and also works with machines on NFS datastores.

SAN reads the disk directly from the storage LUN with no trip over the LAN. The documentation calls it the fastest method for software on SAN-connected ESXi hosts. It needs a physical proxy with access to the SAN storage (Fibre Channel, iSCSI or SAS arrays) and raw-device access to the LUN. It is not supported for machines on vVol datastores, and the library disables SAN when it detects vSAN. NFC session limits do not apply to SAN or HotAdd.

The default priority over the network is san, then hotadd, then nbdssl, then nbd, assuming all are available. If a program asks only for an advanced mode and it is unavailable, the disk open falls back to NBD without failing. (Page text from the VDDK 1.1 and ESXi 5.x era.)

How to prove it

All steps only read. Your backup product decides where its VDDK log is and its log level. Verbose lines may be hidden at a low level.

1. Find the mode lines in the proxy's VDDK log. Broadcom KB 373502 (ESXi 8.x) shows, in this order with other lines between them (disk released, hot-remove successful, temporary hotadd folder deleted):

error Mounting <disk> using transport hotadd failed error Cannot use mode hotadd to access [<datastore>] <vm>/<vm>_1.vmdk: Cannot mount using this method. verbose **** TryMode: Use mode nbdssl to access [<datastore>] <vm>/<vm>_1.vmdk

The first two say which mode was rejected for which disk. TryMode says which mode is tried next, at verbose level in that KB. No fetched page shows a line confirming the mode that finally ran, so use your backup product's job report for that.

2. For HotAdd, open the proxy VM in vCenter. Confirm it sees the target's datastore, the target disks are SCSI, the VMFS version and block size match, and the SCSI controllers were added in numeric order. Note the controller type under Edit Settings.

3. For NBD and NBDSSL, search the host's hostd log for NFC memory messages. Run this on the ESXi host against the hostd log file:

grep -E "NfcCheckAndReserveMem|AIO buffer|Couldn't reserve memory" <hostd log file>

Per KB 443254 and KB 446179 (ESXi 7.0 and 8.0) the cause looks like this:

NfcCheckAndReserveMem: Cannot allocate any more memory as NFC is already using <bytes> and allocating <bytes> will make it more than the maximum allocated: <bytes> Couldn't reserve memory for size <bytes> Cannot allocate AIO buffer, malloc failed

The maximum figure appears only when the limit is hit. If the event is older, check the rotated hostd logs. If none appear, the available logs do not establish NFC exhaustion; retention and logging scope can prevent ruling it out.

4. For SAN, confirm in the vCenter Datastores view that the datastore type is VMFS, not vVol or vSAN, and that the proxy is physical. LUN visibility is checked in the proxy's own disk tool; no fetched page gives a command. The documentation says a Windows proxy should have SAN policy onlineAll.

Documented causes

1. HotAdd rejected because the proxy cannot mount the attached disk

vCenter reports the attach as successful, but the proxy's operating system never mounts the disk. KB 373502 (ESXi 8.x) gives the cause as the legacy LSI Logic SAS controller on a Windows proxy not processing the plug-and-play event needed to mount the disk, and the fix as moving the proxy's disks to a VMware Paravirtual SCSI controller. What singles it out: the proxy's controller type in Edit Settings is LSI Logic SAS and the attach succeeds in vCenter. KB 313507 covers the controller change and says hot add or remove needs a bus rescan in the guest.

Official pages conflict. An older tips page with no stated version says the paravirtual controller is not supported for HotAdd and to use the LSI controller. KB 373502 recommends paravirtual for a Windows proxy on ESXi 8.x. No fetched page reconciles them, and we have not tested either controller type on a proxy.

Other documented HotAdd rejections are IDE disks and SCSI controllers added out of numeric order. KB 440635 describes HotAdd failing with an invalid-parameter error on restored machines with non-persistent independent disks on vVol datastores. The stated cause is a namespace version mismatch: a disk backed up on 7.0 U3 uses version 1, while the restored disk on 8.x defaults to version 2. Fixed in 8.0.3 P10 and 9.1 GA per the KB; NBD still worked, slowly.

2. NFC memory exhausted on the host

NBD and NBDSSL sessions draw from a memory pool in the host's NFC service. When concurrent sessions use all of it, jobs hang at the snapshot stage with "broken pipe" errors and the lines above. The pages give two figures. The NBD best-practices page gives the hostd NFC memory limit as 96 MB, and KB 446179 gives a default maxMemory of 100663296, which is 96 MiB in binary units. KB 443254 shows a maximum of 50331648 in its log, which is 48 MiB, and describes the limit as approximately 48 to 50 MB. The pages do not say why the figures differ. Read the maximum in your own hostd log.

The NBD page recommends 50 or fewer parallel disks per ESXi host. KB 443254 lowers concurrency in the backup product (seven or fewer tasks for NBD in its case) or staggers start times. KB 446179 adds that orphaned sessions can hold disk files open, so a later power-on shows a false locking conflict.

3. SAN unavailable

SAN is not possible when the proxy cannot reach the LUN, is a virtual machine, or the datastore is vVol or vSAN. No fetched page lists SAN failure messages.

Recovery decisions and escalation

Keep the working backup path while investigating a rejected mode, provided it satisfies the product's support, security and recovery requirements. A fallback can be intentional; it is not automatically a fault that justifies changing a proxy or host.

Record the actual VDDK/backup-product version, proxy OS and appliance build, hypervisor build, datastore architecture and selected transport. Use the backup supplier's current compatibility and configuration guidance for that combination. The older SDK pages here explain mechanisms and historical limitations; they do not establish a current operational recipe.

If logs implicate NFC pressure, first review workload and concurrency with the backup owner. Any proposed reduction must still meet the protection window and be measured. Host NFC tuning and service restart require current build-specific supplier guidance, a host workload assessment, management recovery access and post-change protection checks; no tuning value or restart command is prescribed here.

Proxy SCSI-controller changes require confirmed guest-driver availability, boot recovery, product support and a separately recoverable configuration. Windows automount policy must likewise come from the current backup product's supported proxy design. Generic controller migration and automount commands have been withheld because the historical documentation alone does not establish their suitability.

Review transport security as well as speed. The VDDK transport reference distinguishes NBDSSL encryption from NBD; verify certificate/peer-identity validation, network controls and the actual negotiated transport for the installed product. Do not disable verification or accept an insecure fallback merely to complete a test. Encryption alone does not establish verified peer identity.

Acceptance requires one authorised disposable-VM backup and restore with known application data, observed transport, cleanup of attachments and confirmation that other proxy jobs and protection windows remain healthy. If a disk remains attached, use the separately scoped orphaned-proxy decision path rather than deleting disk files or repeatedly consolidating. All configuration and restore checks remain untested here.

What not to do

• Do not silently accept a fallback without recording its support, security, capacity and backup-window implications. A supported network transport may be a deliberate choice.
• Do not turn historical transport-selection advice into a current configuration rule. Confirm the installed product's supported mode selection and fallback policy.
• Do not remove the protected machine while a HotAdd disk is attached, or delete its snapshot before the library cleanup call releases HotAdd disks. Redo logs are left behind.

Transport candidates to validate

The following are candidates, not a compatibility guarantee. Confirm the current backup-product/VDDK support matrix and security design for the exact versions.

• Physical proxy on the SAN fabric: SAN may be eligible.
• Virtual proxy, SCSI disks, target datastore visible: HotAdd.
• vSAN datastores: HotAdd from a virtual proxy, otherwise NBD or NBDSSL; the library disables SAN on vSAN.
• vVol datastores: HotAdd or NBD/NBDSSL; SAN is not supported. The HotAdd fault in KB 440635 is narrow (restored machines with non-persistent independent disks, fixed in 8.0.3 P10 and 9.1) and NBD still worked there.
• Network path only: NBD, or NBDSSL for encryption.

Acceptance checklist

These are required checks, not completed test results.

• [ ] Actual and attempted transport recorded, with rejection evidence.
• [ ] Product/VDDK, OS, appliance, datastore and host versions matched.
• [ ] Current vendor support established before configuration changes.
• [ ] Encryption and certificate/peer verification requirements reviewed.
• [ ] Proxy and host changes have recovery and shared-workload impact plans.
• [ ] Disposable backup/restore and attachment cleanup demonstrated before acceptance.

Sources

1. Advanced Transport APIs, Broadcom developer portal, Virtual Disk API (mentions VDDK 1.1 and ESXi 5.x). https://developer.broadcom.com/xapis/virtual-disk-api/latest/vddkFunctions.6.11.html
2. Virtual Disk Transport Methods, Broadcom developer portal, Virtual Disk API (text for vSphere 5 and 6, VDDK 5.1 to 6.0). https://developer.broadcom.com/xapis/virtual-disk-api/latest/vddkDataStruct.5.5.html
3. Tips and Best Practices, Broadcom developer portal, Virtual Disk API (no version stated). https://developer.broadcom.com/xapis/virtual-disk-api/latest/vddkBkupVadp.9.5.html
4. Windows Backup Implementations, Broadcom developer portal, Virtual Disk API (names VDDK 1.1, 1.2, ESXi 4.1, vSphere 5.1). https://developer.broadcom.com/xapis/virtual-disk-api/latest/vddkBkupVadp.9.6.html
5. Best Practices for NBD Transport, Broadcom TechDocs (vSphere 7.0 path; VDDK 6.5 to 7.0.3). https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/7-0/best-practices-for-nbd-transport.html
6. KB 443254, VDDK async operation error 16000 during an NBD backup. ESXi 7.0, 8.0, 8.0 U3e; VCF 5.2. https://knowledge.broadcom.com/external/article/443254/error-vddk-async-operation-error-16000-d.html
7. KB 446179, ESXi VM power-on fails with ".vmdk locking conflict" due to hostd NFC memory exhaustion. ESXi 7.0.x and 8.0.x. https://knowledge.broadcom.com/external/article/446179/esxi-virtual-machine-poweron-fails-with.html
8. KB 373502, Unable to hot add disk while running a backup. ESXi 8.x. https://knowledge.broadcom.com/external/article/373502/unable-to-hot-add-disk-while-running-bac.html
9. KB 313507, Configuring disks to use VMware Paravirtual SCSI (PVSCSI) controllers. vCenter 7.0, 8.0, 9.0; ESXi 7.0, 8.0; ESX 9.0. https://knowledge.broadcom.com/external/article/313507
10. KB 440635, VM backups with HotAdd transport mode fail on VVOL datastores. vSphere 8.x and 9.x, VCF 9.x. https://knowledge.broadcom.com/external/article/440635