Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

APD or PDL: classify an inaccessible ESXi datastore

Technical article
Troubleshooting

An inaccessible datastore and a list of dead paths do not tell you whether ESXi has received a permanent-loss response. Classify the storage event from the affected device's evidence before explaining why a VM paused, a host became unresponsive or HA did not restart a workload.

This is an inspection-only diagnostic for ESXi 8.0. It does not recover storage or test failover. Examples are synthetic and unexecuted; no customer system was inspected.

Why identical symptoms need different explanations

All Paths Down (APD) means every path is unavailable and ESXi lacks a supported indication that the device is permanently lost. Permanent Device Loss (PDL) means the host has evidence that the device is permanently inaccessible. Broadcom documents both states producing unavailable datastores and dead paths. A11-S1

The distinction is what the host knows. APD is not a promise that connectivity will return. A permanently removed LUN can still present as APD when the array does not supply the supported permanent-loss response. Conversely, PDL does not mean that an administrator should immediately remove every reference to the device.

Storage loss also affects management. Commands waiting on inaccessible storage can contribute to an unresponsive host. A disconnected inventory object therefore does not, by itself, establish a management-network failure.

Assemble a device-specific timeline

Prerequisites are authorised access to existing host logs, the datastore-to-device mapping and cluster HA settings. Use previously collected support material if direct access is unavailable. Do not enable services or restart agents to obtain this evidence.

Record the datastore identifier, backing device identifier, affected hosts and first known failure time. For a multi-device datastore, retain each backing identifier separately. Correlate existing vmkernel.log records with that identifier; a nearby message about another LUN is not evidence for this incident.

The documented SCSI PDL example includes:

H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x25 0x0

Broadcom identifies the 0x25/0x0 additional-sense combination as logical unit not supported and describes explicit permanent-loss messages. Preserve the complete record and its device context. Do not diagnose PDL from an isolated H:0x1 field copied from an unrelated command. A11-S1

For APD, look for explicit APD state transitions or the documented retries with Not found (APD). KB 318712 also includes HPP examples such as HppDeviceUpdateState changing a device to APD and HppIsDevicePDL reporting PDL. Match the actual multipathing plug-in and device; the KB’s NMP strings are examples, not a universal requirement. This does not establish a general NVMe-specific log rule.

The KB’s PDL examples also contain H:0x1 D:0x0 P:0x0. That isolated code does not distinguish the cases: correlate APD Notify PERM LOSS or the documented 0x5 0x25 0x0 sense information for the same device and interval.

Worked interpretation

Consider this invented incident record:

Observation Interpretation
Datastore D is inaccessible on host H1 Confirms impact, not loss classification
Every path for D's device is dead Still compatible with APD or PDL
H1 logs an APD transition for that device Supports APD at that time
Array team reports the LUN was permanently removed Explains the cause but does not rewrite the host's observed state

The report should say “H1 observed APD after the LUN was removed”, rather than converting the record to PDL because the removal was intentional. If a later permanent-loss response appears, record the transition and both timestamps.

A second host might have a different path history. Do not promote one host's conclusion to the whole cluster without checking its peers' evidence.

Interpret HA separately

VM Component Protection distinguishes APD and PDL responses. Inspect the configured cluster response and relevant VM overrides after classifying the storage event. Response settings, timers and restart feasibility matter; detection alone does not prove that HA should have restarted the VM immediately. A06-D1

Keep the observations separate: storage event, VMCP configuration, HA event and resulting VM state. A VM with an inaccessible RDM can also pause awaiting a storage-loss prompt, followed by a Tools heartbeat timeout. That sequence is not sufficient evidence that VMware Tools caused the incident. A11-S1

Acceptance evidence and limits

A useful diagnostic result names the exact device and host, cites a timestamped APD or PDL indication, explains ambiguous evidence and records the applicable HA response. If only “all paths dead” is available, the honest result is unclassified storage loss.

After an independently approved recovery, acceptance should include restored device accessibility and workload-owner confirmation of application health. A green inventory icon alone does not establish consistent application recovery. These are proposed checks, not completed tests.

This inspection changes nothing, so it needs no configuration rollback. Unmounting, detaching, answering a destructive VM prompt, rebooting or re-presenting storage requires a separately validated recovery plan. Broadcom's recovery sections are not safe generic next steps for every APD or PDL event.

Learning notes

  • APD describes uncertainty at the host, not temporary damage.
  • PDL requires device-specific permanent-loss evidence.
  • HA response is a separate decision from storage classification.

References

Related articles