Australian cloud for business software — built and run in Sydney since 2003

NetLeverage
NetLeverage

Blog Articles

Explore 267 articles and one white paper. Choose a topic, then open an article to read more.

Illustration of people writing and creating articles

268 of 268 articles - All topics

Filter articles

Performance

CPU overcommit in hosted clouds: ready time, NUMA and how to size honestly

Overcommit is normal, and some vCPU wait comes with it. It becomes a problem when wait lines up with a real symptom or climbs above your own baseline, so size from measured wait, not from a ratio. None of the VMware, Proxmox or Microsoft documents checked for this article gives a general vCPU-to-core ratio to aim for. They give measurements and warning levels, which is what this article uses.

Read article
Cloud

Ceph for private cloud: replication against erasure coding, latency and recovery load

For virtual machine disks, start with three-way replication: size=3 , min_size=2 , failure domain host. Choose erasure coding only when capacity matters more than small-write latency, you have at least seven hosts for a 4+2 profile, and the cluster is on Tentacle or later so the newer erasure-coding write path is available. There is no in-place conversion: an existing pool means a new pool and moving every object. Whichever you choose, size the cluster for recovery, because that is where the two designs differ most and where an undersized cluster hurts its tenants.

Read article
Backup & Recovery

Availability arithmetic: what 99.9, 99.99 and 99.995 mean when components are chained

Over a 365-day year, 99.9% allows 8.76 hours of downtime, 99.99% allows 52.56 minutes and 99.995% allows 26.28 minutes. In a 30-day month those budgets are 43.2, 4.32 and 2.16 minutes. For independent hard dependencies, component availabilities multiply, so six components that look healthy on their own can deliver about 99.86% together. Redundancy can repair that, but only to the extent that the redundant parts fail independently, and they rarely do.

Read article
Cloud

Stretched clusters: witness placement, split brain and what each failure really does

A stretched cluster needs a product-specific way to decide which site may write after a failure and to prevent competing writers. Witnesses, arbiters and fencing perform different parts of that decision. Place the required arbiter somewhere that fails independently of both sites and has its own path to each. Then test four separate events, because a site loss, a link loss, a witness loss and a double failure do different things in each product.

Read article
Cloud

ESXi to Proxmox VE: disk import, drivers, networking and what does not carry over

Copying the disks is the easy part of leaving ESXi. Most failed moves come from the guest meeting different virtual hardware, so the work is in three places: boot firmware and disk bus, then the VirtIO driver switch on Windows, then guest network identity. Bring Windows up first on SATA or IDE, then switch to VirtIO with a temporary disk. Linux needs its VirtIO drivers in the initramfs before the move.

Read article
Cloud

Proxmox HA with Ceph: quorum, fencing and why nodes reboot

A Proxmox node may reset deliberately after losing quorum while holding active HA services. If its watchdog expires, self-fencing removes the node so another node can safely start those guests. Confirm that chain in the logs before attributing an unexplained reset to HA. The cure is rarely to turn fencing off. It is to find what broke Corosync, and to size votes, monitors and replicas so that one failure cannot take all three down.

Read article
Cloud

Leaving VMware for Hyper-V: failover clustering, CSV and live migration in depth

Moving from vSphere to Hyper-V is a redesign of three things: who holds quorum, which node owns each volume, and how running VMs are allowed to move. Copying the disks is the easy part. A cluster built as a like-for-like vSphere copy can pass a quick test and still go down when one node and a witness fail together, send a whole volume's I/O over a network, or migrate one VM at a time without anyone having chosen that.

Read article
Cloud

KVM and virtio for Windows guests: drivers, disk and network tuning

Windows has no built-in virtio drivers, so the storage driver must be installed and bound to a real device before the boot disk moves to a virtio controller. Get that wrong and the VM stops at boot. Once it runs, the settings that matter most are the driver release, the controller type, IO threads, the cache mode and whether Hyper-V enlightenments are on. Most of the other options should stay at their defaults until a measurement says otherwise.

Read article
Cloud

Mapping dependencies before a cloud migration: flow data, DNS logs and SQL connections

A dependency map built from one data source will be wrong in a predictable way. Flow data shows which addresses talked. DNS logs show which names were asked for. SQL Server metadata shows which clients, jobs and linked servers rely on a database, including those that were silent during the capture. Use all three, then treat every server pair as unproven until a named owner confirms it or a test proves it.

Read article
Troubleshooting

Windows will not boot after migration: diagnose storage and firmware

An INACCESSIBLE_BOOT_DEVICE stop after a migration means Windows could not access its system volume during startup. It does not identify one particular driver, and it does not establish that the disk has failed. The useful first decision is where startup stopped: firmware, boot manager, Windows storage access, or encrypted-volume access. That distinction determines what evidence to collect and which specialist procedure might apply.

Read article
Troubleshooting

Linux boot failure after migration: drivers, identifiers and network names

A Linux VM that stops after migration may have the right data on disk and still be unable to find it. The new hypervisor can change the storage controller, firmware, device path and network identity. The quickest useful diagnosis is to locate the failed stage: firmware, boot loader, initramfs, root filesystem, later mounts or networking. Each stage has different evidence and different repair tools.

Read article
Cloud

Moving SQL Server to the Cloud with Minimal Downtime: Log Shipping, Distributed Availability Groups and Cutover Checks

Two native methods shrink a SQL Server migration outage to the final log handover and the steps after it. Log shipping works on Standard edition and is simple, but you carry the last log backups across by hand. A distributed availability group keeps the target closer and can prove that nothing was lost, but it needs Enterprise edition and an availability group with a listener on each side. If the target runs a newer major version, either method becomes a one-way door after the newer target is recovered or takes over; returning business writes then needs a separate data-reconciliation plan.

Read article
Cloud

Cloud Repatriation: Transfer Time Arithmetic, Data Gravity and Re-Platforming Risk

Leaving a public cloud is limited less by the data transfer bill than by three other things: how long the copy takes while data keeps changing, what the provider will not export, and which managed services have to be rebuilt elsewhere. The major providers now credit transfer charges for customers who are genuinely leaving, on conditions that can shape the plan. Do the arithmetic and the export check before setting a date.

Read article
Backup & Recovery

Kubernetes etcd Backup and Restore: What It Holds and How to Restore a Control Plane

An etcd snapshot restores the Kubernetes API objects and nothing else. To recover a control plane you also need the certificates and any encryption configuration. Permanent quorum loss is a principal disaster-recovery case. An intentional state rollback is a separate decision because it discards newer changes; every API server must be stopped before restoring etcd. Restore every member from the same snapshot, and use a revision bump so that controllers do not act on stale caches.

Read article
Backup & Recovery

Kubernetes Persistent Data: CSI Snapshots, Velero and Application Consistency

A CSI snapshot does not by itself guarantee application consistency, and ordinarily covers one volume unless a supported group snapshot is used. It does not coordinate a database's state, and it may not leave the storage system that holds the original. Velero can add the Kubernetes objects and move the data elsewhere, but application consistency is still your design. Prove it in an isolated restore environment; a separate namespace alone does not isolate network access, external services or cluster-wide resources.

Read article
Backup & Recovery

Immutable Backup Repositories on Linux: Hardened Design, Attributes, Credentials and Limits

An immutable repository is only as strong as the account or system that can lift the lock. On Linux, the i attribute stops root from changing a file, but root can usually clear the attribute first. Real protection comes from making sure no credential reachable from production can get to the process, the host or the storage that holds that power.

Read article
Backup & Recovery

Active Directory Forest Recovery: Authoritative Restore, RID Pool and DC Order

In a forest recovery you restore one writeable domain controller (DC) per domain in isolation, forest root first and parents before children. You restore the directory on that DC nonauthoritatively and SYSVOL authoritatively . Then you seize the operations master roles, remove the other DCs from the directory, raise the RID pool and reset the critical passwords, and only then build the remaining DCs as new ones.

Read article
Backup & Recovery

Proving RPO and RTO: designing restore tests that measure real numbers

A successful backup job proves that a job finished. It says nothing about how much data you would lose or how long you would be down. To get those two numbers, inject a failure while an external writer records every acknowledged write, time each recovery stage from the moment of failure, and stop the clock only when a real application transaction succeeds on the recovered system.

Read article
Backup & Recovery

Bare-metal recovery for Windows Server: VSS, system state and dissimilar hardware

Windows Server has two recovery types that look similar and behave differently. A bare-metal recovery rebuilds disks and the operating system from a recovery environment, and Microsoft supports it on different hardware. A system state recovery runs inside a working Windows installation, and Microsoft does not support it across a different make, model or hardware configuration.

Read article
Backup & Recovery

Backing up NAS and file shares: NDMP, SMB, changed-file detection and permissions

A file share backup has to answer three questions: which path moves the data, how changes are found, and whether permissions come back. NDMP (Network Data Management Protocol) moves data under the storage system's control, and on ONTAP's dump engine it reads from a snapshot. An SMB copy is an ordinary client reading files one at a time, so it inherits every limit of being a client.

Read article
Backup & Recovery

Microsoft 365 Data: What Native Retention Restores, What It Does Not, and Graph API Throttling

Microsoft 365 gives you several short recovery windows and a separate compliance system. Its native rollbacks work per library or per OneDrive for 30 days, and none of these is a tenant-wide point-in-time restore. A deleted mail item is recoverable for 14 days by default and at most 30, while unheld calendar items have a 120-day recovery retention window . A deleted SharePoint item has 93 days across two recycle bins . Retention keeps a copy that is reached through compliance tools , and Microsoft says of holds that they are "optimized for export (for example, via eDiscovery), not for mass restore" .

Read article
Backup & Recovery

Essential Eight regular backups: the technical controls at each maturity level

The regular backups strategy has the same four backup controls at every maturity level, and two access controls (ISM-1812 and ISM-1814) also stay unchanged. What rises between levels is the access limit: each level removes rights from another class of account, until at Maturity Level Three even the backup administrator cannot change or delete a backup during its retention period. A backup design therefore reaches a level through its permission model, its storage locks and a tested restore, not through how often the job runs.

Read article
Cloud

Keeping data in Australia: region controls, key custody, support access and APP 8

Selecting an Australian cloud region does not by itself settle Australian Privacy Principle 8 (APP 8), and encrypting the data does not settle it either. APP 8 asks whether personal information was disclosed to someone overseas. Region policy, key custody and support approvals are the technical evidence for that answer, and they only work alongside the contract.

Read article
Troubleshooting

Missing vCenter History: Retention, Filters or the Wrong Date?

A missing event in the vSphere Client does not prove that a VM was never powered off. First establish the selected object and filters, then compare the incident date with the actual event-retention policy. If the event has been deleted, changing the query tool cannot recreate it. A VM creation timestamp is a different kind of evidence.

This is a source-only diagnostic for vCenter 8.0. No appliance, database or event store was inspected for this article. The examples are hypothetical.

Read article
Troubleshooting

vCenter Alarm Acknowledged but Still Red: What the State Means

Acknowledging a vCenter alarm records ownership and stops its alarm actions; it does not clear the alarm. A red or yellow state after acknowledgement can therefore be expected. For an event-triggered alarm, the state can also remain after the incident ends if vCenter never receives the corresponding normal-condition event. Resetting its colour is a different operation from repairing the cause.

These distinctions come from the vSphere 8.0 acknowledgement and event-reset guides linked below. This article is source-researched only. No alarm, rule or cluster configuration was changed, and its example is hypothetical.

Read article
Troubleshooting

ESXi Logs Vanish After Reboot: Check the Active Scratch Location

Read the active scratch location together with the syslog destination. A configured persistent directory does not prove that ESXi has started using it. If logging still resolves to a ramdisk, a reboot can remove the very evidence needed to explain the previous failure.

This diagnostic focuses on ESXi 8.0.x. KB 317690 lists USB/SD boot media smaller than 32 GB in its environment (7.0.x/8.0.x); its logDir/scratch mechanism is used here with that boot-media context. Sources are KB 317690 and KB 302451. It is source-only: no host was inspected, no probe executed and no reboot performed.

Read article
VMware

Subscribed Content Library: Template Listed, Payload Not Downloaded

A synchronised subscribed Content Library can contain an item's metadata without its complete payload. If the library downloads content only when needed, the presence of the item in the catalogue is not evidence that its files are available locally for an offline deployment. Check the download mode and the individual item's content state before treating a small size as corruption.

This source-only article uses the vSphere 8.0 Content Library guide. No library was synchronised, content downloaded or VM deployed. The worked example is fictional.

Read article
Troubleshooting

Linux VM Customisation Sets the IP but Skips Your Script

Check the VMware Tools custom-script setting and the phase your script expects before rebuilding the template. A Linux guest can have running Tools and receive its normal customisation settings while the optional script is disabled. A script that does run can also fail because it expects networking during a phase in which VMware deliberately disconnects the virtual NIC. Broadcom KB 313048 documents both boundaries.

This is a source-only diagnostic for vSphere 8.0.x Linux guest customisation. No VM was cloned, script executed or configuration changed for this article. The examples below are hypothetical, not observed customer results.

Read article
VMware

FT Secondary Will Not Start: Check the VMCP Policy Error

A Fault Tolerance secondary that cannot power on is not automatically a capacity or FT-network problem. In the vSphere 8.0.x case documented by Broadcom KB 451614, the HA agent rejects the operation because VM Component Protection, or VMCP, is configured incompatibly with the FT workload. The decisive evidence is the policy message for that VM, not merely a red FT status.

This article explains that exact diagnostic branch. It is source-researched and untested; no host, VM, HA setting or protection policy was changed.

Read article
Troubleshooting

One vNUMA node, yet CPU Hot Add blocks VM reconfiguration

An ESXi 8.0 GA VM can report “Virtual NUMA cannot be configured when CPU hotadd is enabled” even with one vNUMA node. Broadcom documents a configuration-validation problem involving virtual hardware 20, an ACPI motherboard layout, CPU Hot Add and a PCI passthrough device after a power cycle.

This inspection-only guide explains the documented case without assuming a physical NUMA boundary was crossed. ESXi 8.0 GA predates U1; later-build incidents need separate investigation. No reconfiguration or performance test was executed.

Read article
Troubleshooting

VM Power-On Fails Creating Swap Despite Spare Host RAM

Spare host RAM does not eliminate the VM's swap-backing requirement. ESXi reserves swap space for the difference between configured VM memory and its memory reservation. A power-on can therefore fail because the required swap file cannot be created on its selected datastore, even before the guest starts using memory.

This article covers the ESXi 8.x error family in KB 341571. The KB also lists ESXi 7.x, but this article intentionally uses the 8.0 resource guide and does not validate a 7.x procedure. No VM was powered on and no reservation or storage configuration was changed. All calculations below are hypothetical.

Read article
Performance

Equal Resource Pool Shares Do Not Mean Equal CPU per VM

Resource-pool shares first divide resources between sibling pools, then the VMs inside each pool compete within that allocation. Two equally weighted pools do not automatically give every VM beneath them equal entitlement. A VM in a busy five-VM pool can therefore have a different relative entitlement from the only VM in the other pool, despite both pools displaying Normal.

This is a source-researched explanation of vSphere 8.0 CPU shares. No workload test or resource-pool change was performed. The arithmetic is an explicitly simplified example, not a prediction of application throughput.

Read article
Troubleshooting

Same NFS export, different datastore: compare ESXi mounts

If two ESXi hosts reach the same NFS 3 export but vSphere treats it as different datastores, compare the configured server and folder strings. Broadcom explicitly requires identical server and folder names across hosts mounting the same NFS 3 volume. A short server name on one host and a fully qualified name on another can create distinct datastore identities even when both resolve to the same server. A10-D5

This diagnostic uses the vSphere 8.0 storage documentation. It reads existing configuration only. Examples are invented, no commands or migrations were executed, and no storage remount is proposed.

Read article
Troubleshooting

APD or PDL: classify an inaccessible ESXi datastore

An inaccessible datastore and a list of dead paths do not tell you whether ESXi has received a permanent-loss response. Classify the storage event from the affected device's evidence before explaining why a VM paused, a host became unresponsive or HA did not restart a workload.

This is an inspection-only diagnostic for ESXi 8.0. It does not recover storage or test failover. Examples are synthetic and unexecuted; no customer system was inspected.

Read article
VMware

ESXi TPM recovery: a host backup is not the recovery key

A configuration backup alone does not establish that an ESXi host can recover its encrypted configuration after a TPM or motherboard change. First identify the host's configuration-encryption mode and confirm that the matching recovery material has an authorised custodian. Do not substitute a VM key-provider backup or a Windows recovery password for the host's own requirement. A12-S1, A12-S2

This is an inspection-only assessment for ESXi 8.0, using Broadcom's guidance for host configuration encryption introduced in the 7.0 Update 2 era. No hardware maintenance, recovery boot or key retrieval was performed.

Read article
VMware

Why an existing distributed port works while a new one fails

A vCenter outage does not automatically stop traffic through existing static-bound distributed ports. It can prevent an administrator from allocating or changing a binding needed by a new or reconfigured VM. Treat those as different operations before concluding that the distributed switch has failed. A13-S1

This inspection-only explanation covers vSphere 8.0 static and ephemeral distributed port groups. No VM was deployed, powered on or moved, and the example was not executed.

Read article
Troubleshooting

NSX Logs Late TCP Packets Against the Wrong Firewall Rule

A FIN or RST packet logged against a lower NSX Distributed Firewall rule does not necessarily mean the connection's original allow rule failed. In the stateful TCP case documented by Broadcom KB 343357, the connection may already have left the flow table. Its late non-SYN packet is then evaluated differently from the SYN that created the original flow.

This article covers that specific NSX 3.x/4.x behaviour. No firewall, flow table or logging configuration was changed, and no packets were generated. The example is synthetic.

Read article
VMware

NSX Tier-1 Segment Has No Upstream Route: Trace Advertisement

Check whether the Tier-1 advertises the connected segment to Tier-0, then check how the physical network routes back to that subnet. A VM reaching its local gateway does not establish either relationship. A Tier-0 default route toward the physical router supplies an outward path; it is not a return route on that router.

This article uses the NSX 4.2 static-routing example in Configure Gateways and Segments. No NSX or physical-network configuration was inspected or changed for this source-only research, and no connectivity test was performed.

Read article
Backup & Recovery

Why a completed replication can still violate its RPO

A completed synchronisation can leave an RPO violation because its recovery point represents the VM when synchronisation started, not when transfer finished. Compare the age of the latest available instance with the configured Recovery Point Objective (RPO), then investigate why newer instances become available too late.

This article uses the vSphere Replication 9.0 documentation. It is source-only analysis: the timeline is illustrative, and no replication, recovery or performance test was run.

Read article
VMware

A healthy vSAN witness cannot replace missing VM data

A connected witness does not prove that a vSAN stretched-cluster object has enough usable data to remain accessible. The witness stores metadata that helps vSAN decide availability; it does not hold another copy of the VM's user data. Diagnose the affected object's surviving components and failure sequence alongside witness connectivity. A17-P1, A17-S2

This explanation follows the current official stretched-cluster guide, which explicitly targets vSAN 9.1 in VMware Cloud Foundation 9.1 and primarily discusses ESA. It is an inspection-only article, not a VCF deployment or recovery procedure. No site failure, object repair or takeover was executed.

Read article
Troubleshooting

Diagnose a vCenter PNID and hostname mismatch safely

When a vCenter pre-check says its Primary Network Identifier (PNID) does not match its hostname, compare the two exact values before changing certificates or directory settings. Broadcom documents mismatches that differ only in letter case. A name resolving successfully in DNS does not establish that vCenter's stored identity and operating-system hostname agree. A18-S1, A18-S2

The scope is vCenter Server Appliance 7.x and 8.x. These are source-checked, unexecuted read-only examples. No diagnostic tool, appliance command or identity change was run for this article.

Read article
VMware

Storage DRS Has Free Space but No Legal Disk Placement

Free space elsewhere in a datastore cluster is not sufficient for Storage DRS to move a VM's disks. The destination must satisfy placement constraints, and the VM's automation settings must permit the proposed operation. In vSphere 8.x, intra-VM disk affinity and VM-level Manual or Disabled overrides are two documented reasons for apparently unbalanced storage.

This source-only diagnostic follows KB 432448. No storage policy, recommendation or VM was changed or tested. The numerical example is hypothetical.

Read article
Troubleshooting

VASA Connected in vCenter, but ESXi Reports INVALID_SESSION

A VASA provider showing Connected in vCenter does not prove that every ESXi host has a usable session with it. Broadcom KB 459604 documents an ESXi 8.x case in which the provider invalidates its current session without communicating that change to ESXi. Hosts continue using a rejected session, and vVol operations fail while vCenter still shows the provider connected.

No provider, host, certificate or VM operation was changed for this source-only article. The commands below are unexecuted read-only examples; check exact installed help and authorised access before use.

Read article
Troubleshooting

vDS Says a VLAN Is Unsupported on an Uplink the Port Group Does Not Use

A vSphere Distributed Switch health check can test a VLAN on an uplink that a particular port group is deliberately configured not to use. Its warning can therefore describe a real unsupported VLAN/uplink combination without proving that the current workload is using that combination. Compare the probe result with the effective port-group uplink policy before deciding whether the design is broken.

This is a source-only interpretation of vCenter 8.x behaviour. No health check was enabled or disabled, no packet probe run and no switch configuration changed. The example is hypothetical.

Read article
Troubleshooting

A new vDS LAG exists but carries no VM traffic

Creating a link aggregation group (LAG) on a vSphere distributed switch does not automatically move a host's physical NICs or a VM port group's traffic into it. The vSphere 8.0 documentation states that a newly created LAG has no physical NICs assigned and is unused in distributed port groups' teaming and failover order. Inspect those separate states before treating the LAG as a working traffic path. A22-D3

This is an inspection-only diagnostic for enhanced LACP on a vSphere 8.0 distributed switch. No uplink was moved, LAG activated, switch reconfigured or bandwidth test executed.

Read article
Performance

VMXNET3 receive-buffer drops: compare counters before tuning

Rising VMXNET3 receive-buffer exhaustion counters can explain packet drops inside the virtual receive path even when the physical link is healthy. Compare a correctly identified VM NIC over one uninterrupted observation interval, then correlate CPU scheduling and guest processing evidence before increasing buffers. A23-S1

This article covers the Windows VMXNET3 case documented for ESXi 8.0. Record the installed guest driver and VMware Tools builds. It is source-only diagnosis; no packet generator, workload test, migration or configuration change was executed.

Read article
Performance

Investigate Windows PVSCSI Event 129 before queue tuning

A Windows pvscsi Event 129 reports a storage reset, not a complete diagnosis. Before increasing queue depth, correlate the guest timeout with ESXi records and determine whether the evidence supports virtual-controller queue pressure, backing-storage delay or an unresolved combination.

The scope is a Windows guest using VMware Paravirtual SCSI on ESXi 8.0.x. Record the installed PVSCSI driver and VMware Tools versions; the device name alone does not establish the driver build. This is an inspection-only article with synthetic examples and no executed workload tests.

Read article
Troubleshooting

ESXi logs locally but nothing arrives at remote syslog

Local log files can keep growing while an ESXi host sends nothing to a remote syslog collector. Local storage and remote forwarding have separate configuration. Start by reading the forwarding destination and the host's existing syslog errors, then correlate with the receiver's records. Do not change local retention to fix an unproven remote-delivery problem. A25-S1

This article covers ESXi 8.0 and explicitly identifies a behaviour change at 8.0 Update 2b. All command examples are source-checked but unexecuted. No syslog marker, network probe, firewall change or service reload was performed.

Read article
Backup & Recovery

VAMI SFTP backup cannot create a directory: read the first failure

“Failed to create backup directory” does not prove that a vCenter backup account lacks write permission. The adjacent CompleteFTP case presents as a directory-reading failure, not that same creation error. In documented SFTP failures, the directory error wraps an earlier SSH-session problem or a destination-specific response to a missing path. Read the first meaningful failure in the same backup attempt before changing the repository. A26-S1, A26-S2

This article covers vCenter Server 8.x appliance file-based backup diagnosis. The CompleteFTP example is bounded to the documented 8.0.x case. No backup, SFTP connection, script patch or service operation was executed.

Read article
Troubleshooting

Terraform sees a VM IP but the VMware network waiter times out

A visible guest IP address does not necessarily satisfy the vSphere Terraform provider's network waiter. In provider v2.12.0, the default waiter also considers gateway-related information, and configured exclusions can reject an otherwise visible address. Diagnose those predicates before extending the timeout or disabling the gate. A28-S1

This article reads the version-pinned resource documentation. All examples are synthetic and unexecuted. No Terraform plan, apply, refresh, import or guest command was run.

Read article
VMware

DRS pauses when the only Embedded vCLS host is unavailable

In a mixed-version cluster managed by vCenter 8.0 U3, DRS can become temporarily unavailable when the only host capable of running Embedded vCLS becomes unavailable. vCenter may need to fall back to External vCLS, and that transition is not instantaneous. Use the cluster's vCLS events and host timeline before treating disappearing service VMs as accidental deletion. A29-S1

This is a source-only diagnostic for the transition between pre-8.0-U3 ESXi hosts and ESXi 8.0 U3 hosts. No upgrade, maintenance operation, vCLS deletion or DRS test was performed.

Read article
Troubleshooting

VMRC opens but cannot connect: check the client-to-host path

If VMRC opens but reports “Invalid or expired session ticket” or “No response” while the web console works, inspect the VMRC workstation's path to the ESXi host. For the default MKS console transport in VMRC 11.x and 12.x, Broadcom documents a direct WebSocket connection to ESXi over TCP 443. Reaching vCenter in a browser does not prove that path works. A30-R1, A30-R2

This article is an inspection-only explanation of existing evidence. No console, connection probe, firewall change or client preference modification was executed.

Read article
VMware

vCenter Admin Group Member Denied an Action: Check for a Restricting Permission

An account is a member of a group that holds the Administrator role, yet in the vSphere Client some actions are greyed out or refused. Group membership alone does not decide what a user can do. vCenter looks at which permissions exist on which objects and for which principal, and some of those can override the group's role.

This article shows how to check, for one user and one VM, whether a documented restriction explains the denial: a permission set directly on the user, or a lower assignment that overrides the group's role.

Read article
Backup & Recovery

SRM Test Would Use the Recovery Port Group, Not the Isolated Network

Before a recovery plan test, someone notices that the plan's test network shows a real disaster recovery (DR) port group rather than an isolated test network. The site's network mapping says Isolated network (auto created) for tests, so how can the plan point somewhere else?

The plan has its own setting. Broadcom documents this exact case in KB 375276:

Read article
Troubleshooting

DRS Moved a VM Back After a Manual vMotion: Rule, Setting or Load Balancing?

A manual vMotion completes, and later the cluster migrates the same VM again, sometimes straight back to where it was.

One documented cause is that your move broke a rule DRS is allowed to repair. vSphere lets a user-initiated vMotion break a Keep Virtual Machines Together or Separate Virtual Machines rule, and a fully automated DRS cluster then remediates the placement (Broadcom KB 393104). The vSphere 8.0 guide documents the same for preferential ("should") VM-Host rules (Using Affinity Rules without vSphere DRS).

Read article
Troubleshooting

NSX Traceflow Says Delivered but the Application Still Fails: What the Trace Proves

A Delivered observation in NSX Traceflow means that a marked test packet built by NSX Manager was reported as delivered at a particular endpoint in the trace. Which endpoint that is has to be checked. When the last observation names the destination VM's logical port, the trace shows that the packet crossed the NSX components it recorded and reached that port. When it names a bridge or another element, it does not show that, as explained below.

Even in the best case, Delivered does not show that the destination operating system received anything:

Read article
VMware

VCF 5.2 Precheck Shows an Old Version After an Out-of-Band Update

Someone patched ESXi, vCenter or NSX directly instead of through SDDC Manager. Now an upgrade precheck in VMware Cloud Foundation (VCF) 5.2 fails, and the version it quotes is the old one, even though the component is already running the newer build.

The cited sources describe at least two candidate explanations. They are not the only possible ones, and more than one can apply at the same time:

Read article
VMware

vDS Host Shows Out of Sync: Trace the Listed Ports First

A vSphere Distributed Switch (vDS) shows the warning "The vSphere Distributed Switch configuration on some hosts differed from that of the vCenter Server", and one or more hosts show Out of sync in the vDS Status column. The usual reflex is to click Rectify, or to remove the host from the switch and add it back.

Broadcom's articles show that this warning has several different causes, and some of them are not fixed by either action. This guide shows how to record what the warning actually names, so the owner can pick the documented path for that cause. It changes nothing.

Read article
VMware

VM Network Adapter Disconnected After a Reboot: Read Its Device State

A virtual machine (VM) comes back from a reboot, a host reboot or a vSphere HA restart, and its network adapter shows as disconnected in the vSphere Client. The quick fix is to tick Connected and move on.

Ticking Connected is a change. It does not tell you why the adapter came up disconnected or whether it will happen again, and the checkbox alone does not show that the guest's network or its applications have recovered. Broadcom documents at least three different causes: a cleared setting, a port the host could not find, and a timing issue on ephemeral port groups.

Read article
VMware

Software iSCSI Paths Dead After a NIC Change: Check the 1:1 Uplink Policy on Bound Ports

An ESXi 8.0 host uses the software iSCSI adapter with port binding. Two or more VMkernel ports sit on one vSwitch, in the same subnet as the array's target portals. Someone changes the host's networking: a failed physical NIC is replaced, a new NIC is added to the switch, or the teaming order of a port group is edited. Afterwards some paths to an iSCSI device show Dead, or a rescan takes much longer than it used to.

Port binding depends on each bound VMkernel port having exactly one active physical adapter. A NIC added to the switch can quietly change that.

Read article
VMware

NSX Tier-0 BGP Neighbour Down: Read the Session State Before Resetting Anything

An NSX 4.2 Tier-0 gateway peers with an external router over BGP. An alarm says a BGP neighbour is down, the Tier-0 BGP neighbour list shows a status other than up, or routes from the physical network have gone. The usual reflex is to clear the BGP session or bounce the uplink. A reset replaces the state that tells you which side failed, and it can drop routes that are still working on other neighbours.

This guide shows how to record one neighbour's session on each Edge node, read what the state means, and match it to Broadcom's documented causes.

Read article
VMware

vSAN Cluster Partition Warning: Record Membership on Every Host Before Anyone Rejoins It

vSAN Skyline Health shows the cluster partition check in red, an alarm says a host is partitioned, or virtual machines on one host have become inaccessible. Broadcom's knowledge base contains repair procedures that make a host leave and rejoin the cluster, rebuild unicast lists, move vSAN traffic to another adapter or reboot hosts. Each of those changes cluster membership or the storage network while data may already be at reduced availability.

This article is the step before anyone considers a repair: record what every host believes about membership, list every documented candidate cause the record supports, and hand that record to the cluster owner or Broadcom.

Read article
VMware

Datastore "Is in Use" and Will Not Unmount: Record the Consumers Before You Change Anything

A storage migration is nearly finished. The virtual machines have moved, the old VMFS datastore looks empty, and the unmount fails with "The resource … is in use" or, from the ESXi command line, "cannot be unmounted. Reason: Busy". The knowledge base fixes for this error include stopping processes, restarting host agents, disabling vSphere HA, enabling vCLS Retreat Mode and rebooting hosts. Several of those affect a whole host or cluster, not just the one datastore.

This article produces a record: each candidate consumer, on each host and each extent, with how strong the evidence for it is and who owns it.

Read article
VMware

VM Shows Orphaned, Invalid or Inaccessible: Record the Evidence Before Anyone Re-registers

After a power event, a vCenter restore or an interrupted migration, some VMs show in vCenter with (orphaned), (invalid) or (inaccessible) after their name. The knowledge base articles for these states end in actions such as removing the VM from inventory, registering or reloading it, stopping its process, reconnecting hosts or editing the vCenter database. None of the three labels picks one of those actions by itself. An Invalid VM may later need a supported reload or registration once its real cause is fixed, and an Orphaned VM may be running normally on another host, where registering a second copy would be the wrong change.

Read article
VMware

Maintenance Mode Stalls: "vSphere HA admission control is preventing the operation"

A host is put into maintenance mode for patching or hardware work. The task stops moving, the virtual machines stay where they are, and the cluster's DRS Faults page shows the line in the title. Admission control is applying the failover capacity rule it was set up with, and refusing migrations that would break it.

A common suggestion is to turn admission control off. That stops vCenter enforcing the failover capacity reserve; vSphere HA itself stays on. Even with admission control on, a restart after a host failure still depends on placement, storage and network, so neither setting is a promise that every VM restarts.

Read article
VMware

ESXi Host Disconnected After a Custom Certificate: Record the Certificate Mode and the Host Certificate

An ESXi host can disconnect from vCenter after a custom certificate is installed. Before changing anything, record the host certificate and vCenter certificate mode, then compare the evidence with the relevant Broadcom case.

This guide explains why different vendor procedures are not interchangeable. Legacy thumbprint mode is intended for debugging and does not check certificate validity. Any certificate-mode change is a security decision; the article gathers evidence for that decision rather than prescribing a general repair.

Read article
VMware

ESXi Took a Network Port Down: "Down by linkflapping" and the Evidence to Collect

vCenter raises an uplink redundancy alarm, and one physical port (vmnic) on the host shows down. A down port alone does not say why. When the host logs "Taking down physical NIC vmnic## because the link is flapping." and the port reports "Down by linkflapping", ESXi switched it off itself because the link kept dropping and coming back.

That protection is deliberate. Bringing the port back up, or turning the protection off, would return an unstable link to service. This article records what the host and the switch saw, so the people who own the cable, the switch and the adapter can find the fault.

Read article
VMware

Storage vMotion Is Slower Than Expected: Record Whether the Copy Was Offloaded

A VM's disks are moved to another datastore with Storage vMotion. It takes hours longer than the change window allowed. The first question is not how to make it faster, but who did the copying: the storage array, or the ESXi host.

On block storage, ESXi can ask the array to copy the data itself. The vSphere 8.0 guide calls this full copy: "Storage arrays can make full copies of data within the array without having the host read and write the data." When that cannot happen, the host reads and writes every block.

Read article
VMware

Host Profile Still Non-Compliant for the Root SSH Key: Record the Difference

The host was remediated against its host profile. The compliance check runs again and the host is still Non-compliant, for one item: "SSH public key not present in profile for root". Remediating again is the obvious move. On ESXi 8.0 it can leave the host exactly where it was, or leave the key in place but unusable for login.

This article records what differs between the profile and the host and matches it to the two documented ESXi 8.0 causes. The owners decide what happens next.

Read article
VMware

HCX Bulk Migration Stalled at START_OFFLINE_SYNC: Record Replication and Switchover Separately

A VM's Bulk Migration showed its data copied days ago. The switchover window opened, and the migration has not moved since. The HCX Manager log says the switchover is running in START_OFFLINE_SYNC.

Two things are easily mixed up here. Replication is the copying of the VM's disks, which happens over days before the window. Switchover is the short sequence inside the window: power off, a last sync, then build the VM at the target. A stall in switchover does not mean the replicated data is lost, and good replication figures do not mean the switchover can finish.

Read article
Troubleshooting

SQL Requests Are Waiting: Find the Head Blocker Before Ending Sessions

When SQL requests wait behind another session, identify the blocking chain and its transaction owner before ending connections. The session with the longest wait is often the customer of the problem, not its cause. A useful diagnosis connects the waiting request, the resource it needs, the session holding that resource and the application operation that should release it.

This guide uses SQL Server 2022. It provides a read-only starting point and a decision process, not an automated session-termination procedure. The examples were checked against Microsoft documentation but have not been executed in a laboratory or on a customer instance.

Read article
Troubleshooting

SQL Deadlock Error 1205: Read the Graph and Fix the Access Pattern

Error 1205 means SQL Server selected a transaction as a deadlock victim. Read the captured deadlock graph before changing timeouts, indexes or retry settings. The graph describes a cycle of resource ownership and waiting; it gives you a much stronger starting point than the final statement reported by the application.1

The practical objective is to reduce the conflicting access pattern and handle any remaining victim errors safely. A successful retry does not demonstrate that the conflict is harmless, and changing which transaction loses does not remove the cycle.

Read article
Performance

A SQL Query Became Slow Overnight: Compare Plans with Query Store

A query that became slow overnight needs a before-and-after comparison, not an immediate server-wide tuning change. Query Store can retain plans and execution statistics across time, allowing a DBA to ask whether a different plan coincided with the regression and whether the work performed was actually comparable.1

Start by confirming that the required history was captured. Then compare the same query, workload interval and execution outcome. A different plan is evidence to investigate; it does not by itself prove why the user experience worsened.

Read article
Troubleshooting

tempdb Is Filling Up: Separate Version Store, Objects and Spills

When tempdb is filling, first determine whether the space belongs to user objects, internal work or row versions. Then identify the workload that creates or retains it. Adding files, restarting SQL Server or shrinking tempdb before making that distinction can remove evidence while leaving the original pressure unchanged.

The reference version is SQL Server 2022. This article explains source-checked, read-only observations and the decisions they support. It does not execute diagnostics, change file sizes or terminate transactions. SQL Server 2025 adds tempdb behaviour that must not be assumed on the 2022 reference system.

Read article
Performance

SQL Queries Wait for Memory: Diagnose RESOURCE_SEMAPHORE

RESOURCE_SEMAPHORE can mean a query is waiting for execution memory. Identify the waiting requests, the grants already held and the workload that consumes them before increasing server memory or imposing a query hint. The immediate symptom is a queue; its cause may be excessive grants, underestimated work or too many legitimate requests running together.1, 2

This article uses SQL Server 2022. Diagnostic examples are source-checked but unexecuted. It does not change memory settings, capture a production execution plan or run a workload benchmark.

Read article
Troubleshooting

SQL Collation Conflicts After a Move: Find the Comparison Boundary

A collation conflict after a server move usually requires identifying the two character expressions being compared, not changing the whole instance. Record the database and column collations, the current query context and any temporary-table definitions. Then choose the comparison rule that matches the application's intended meaning.1, 2

The reference environment is SQL Server 2022 with ordinary non-contained databases. This article does not cover contained-database or cloud-specific catalogue behaviour. Its examples only inspect metadata; no collation change or customer query was executed.

Read article
Performance

Raise SQL Database Compatibility Level with a Measured Query Baseline

Treat a database compatibility-level increase as a measured application change. Establish a representative baseline at the existing level, confirm Query Store is collecting the required evidence, then compare correctness and workload performance after the proposed change. An engine upgrade and adoption of new optimiser behaviour are related decisions, but they are not the same setting.1, 2

This article uses SQL Server 2022 and a possible move from compatibility level 150 to 160 as its reference case. It supplies planning and read-only inspection, not a production change command. No compatibility setting, query plan or application workload was changed or tested for it.

Read article
Troubleshooting

SQL Agent Reports Success but the Import Failed: Audit Job-Step Outcomes

A successful SQL Server Agent job can still have followed a path that skipped required work or treated a failed step as success. Inspect the job's step-level history and its success/failure routing before rerunning the import. The final status is the result of configured control flow; it is not independent evidence that the intended business output exists.1, 3

This article investigates one job's outcome. It does not repeat the separate migration problem of two servers executing the same schedule. Its examples are inspection and design guidance only; no job, import or failure-injection test was run.

Read article
Backup & Recovery

Ad Hoc SQL Backups: Protect the Differential Base with Copy-Only

For an extra full SQL Server backup that should not replace the scheduled differential base, use the native copy-only design after confirming the backup owner's requirements. A copy-only full backup does not become a differential base and does not change the existing one. It can still be restored as a full backup.1

The decision is about recovery dependencies, not the name or location of the file. Copying an ordinary backup to another folder does not make it copy-only, and adding “copy” to a filename does not establish how SQL Server created it.

Read article
Backup & Recovery

SQL Failure Before the Last Log Backup: Is a Tail-Log Backup Possible?

Before restoring over a failed SQL Server database, determine whether unbacked-up log records can still be preserved. A tail-log backup may extend recovery beyond the latest existing log backup, but it depends on recovery model, database state and the condition of the log. Do not overwrite the original database or change its recovery model while that decision is unresolved.1

The purpose is to preserve an option, not to promise recovery of every recent transaction. This article is a SQL Server 2022 incident decision guide. It supplies no executable tail-log or restore command, and no damaged database or laboratory restore was tested.

Read article
Backup & Recovery

Recover Related SQL Databases to a Common Marked Transaction

Related SQL databases need a common logical recovery boundary when their records must agree. A deliberately coordinated marked transaction can provide such a boundary, provided its mark exists in every participating database's usable log sequence. Restoring databases to similar clock times is not a substitute for that coordination.1, 2

Marks must be designed and recorded before the incident. They cannot be added retrospectively to missing history. Recovery to a chosen mark also discards later committed work from the restored state, so the business owner must accept and reconcile that consequence.1

Read article
Backup & Recovery

SQL User Databases Survived but the Instance Is Gone: Plan System-Database Recovery

User database backups do not reconstruct an entire SQL Server instance. If the instance is lost, recovery also needs system-database state, a compatible engine installation and separately preserved configuration, access and key dependencies. Plan these together before deciding whether to restore system databases or reconstruct the instance.1, 2

The goal is a usable, controlled service. An instance that starts but lacks its jobs, identities or required external dependencies is only an intermediate result. Conversely, restoring old scheduled work too early can cause unintended production actions.

Read article
Backup & Recovery

PostgreSQL WAL Archiving Failed: Find the Gap Before Retention Removes It

A PostgreSQL base backup does not establish point-in-time recovery unless the required write-ahead log, or WAL, is also available. When archiving fails, identify the missing segment and the usable recovery boundary before retention or manual cleanup removes the remaining evidence. Restore the archive's correct success behaviour, then verify retrieval and recovery.1

This article uses PostgreSQL 18. It concerns native continuous archiving around a physical base backup, not a logical SQL dump or a VMware freeze/thaw hook. No archive command, backup, restore or server configuration was executed for it.

Read article
Troubleshooting

PostgreSQL pg_wal Keeps Growing: Identify the Replication Slot Holding It

When pg_wal keeps growing, identify whether a replication slot is retaining WAL for a consumer that is no longer advancing. Record the slot's owner, type, required position and current usability before deciding to restore the consumer, provide capacity or deliberately reinitialise it. Dropping an unfamiliar slot to recover space can destroy the consumer's continuation path.1, 2

This guide uses PostgreSQL 18. Some columns and settings differ on older releases. It provides read-only examples and decision boundaries; no slot was dropped, advanced or recreated, and no checkpoint or replication test was run.

Read article
Performance

PostgreSQL Tables Keep Growing: Diagnose Autovacuum and Transaction Age

Growing PostgreSQL tables need an explanation of row-version retention and vacuum progress before a rewrite is considered. Check whether obsolete rows can be removed, whether autovacuum is keeping up and whether transaction-ID ageing creates an urgent protection issue. A large relation file after vacuum does not automatically mean vacuum failed.1

This article uses PostgreSQL 18. Its examples read statistics and progress only. No vacuum, table rewrite, transaction termination or tuning change was executed.

Read article
Backup & Recovery

A PostgreSQL Dump Restored with Missing Roles or Extensions: Check the Recovery Set

A PostgreSQL logical dump can contain the database objects and data while omitting the cluster-wide roles or installed extension files needed to use them. Treat recovery as a set of dependencies: the dump, appropriate global definitions, compatible tools/server, extension support files and an explicit ownership model.1, 2, 4

A restore that reports errors and leaves some tables visible is not a completed recovery. Inspect the archive, preserve every error and validate application behaviour before enabling the destination.

Read article
Backup & Recovery

MySQL Point-in-Time Recovery: Match the Backup to Binary Log Coordinates

MySQL point-in-time recovery needs a consistent backup, its correct binary-log starting boundary and every required event through the chosen stopping boundary. Establish those three facts before replaying anything. A recent dump plus a directory of binary logs is not automatically a coherent recovery set.1, 3, 4

This guide uses MySQL 8.4 and a logical full-backup design. It explains the recovery contract and evidence, not an executable restore pipeline. No dump, binary-log replay or database test was performed. MariaDB and other MySQL-compatible products require their own version-specific assessment.

Read article
Performance

MySQL Replica Lag: Separate Receiver Delay from Apply Delay

When a MySQL replica falls behind, determine whether the delay is in receiving transactions or applying them. Inspect channel state, positions, errors and worker progress together. Seconds_Behind_Source = 0 alone does not prove the replica is current with the source.1

This guide uses MySQL 8.4 asynchronous replication. It does not cover promotion or Group Replication membership decisions. The examples are read-only and untested; no replication thread was stopped, restarted or reconfigured.

Read article
Troubleshooting

Linux Says No Space Left but Bytes Are Free: Check Inodes and Quotas

If an application cannot create a file while a disk report shows free bytes, first verify the filesystem behind the failing path. Then check inode availability and the limits applying to the actual service identity. Free capacity on a different mount, or free blocks outside an enforced quota, does not answer whether that operation can succeed.1, 2, 3

This guide focuses on Linux with GNU df, util-linux findmnt and local ext4/XFS storage. The examples inspect existing state only and were not executed. Network filesystems, container storage and other filesystem types need their own allocation and quota rules checked.

Read article
Troubleshooting

Files Deleted but Linux Disk Space Is Still Used: Find Open File References

Deleting a Linux filename does not necessarily release its storage immediately. If a process still has the last unlinked file open, the file remains until the remaining open references close. Identify the owning process and file before choosing the service's supported reopen or restart procedure.1

This is a specific explanation for space retained after deletion or log rotation. It is not the only reason filesystem usage differs from a directory total. The examples are source-checked and unexecuted; no process was signalled, restarted or inspected on a live service.

Read article
Troubleshooting

LVM Thin Pool Almost Full: Check Data and Metadata Separately

A thin pool has two separate capacity limits: space for data and space for the metadata that maps thin volumes to physical blocks. Check both on the pool itself. A guest filesystem showing free space, or a thin logical volume showing a large virtual size, does not establish that the shared pool can accept another write.1

Pool extension, metadata repair and snapshot removal require a separate approved procedure for the installed environment.

Read article
Troubleshooting

A Linux Filesystem Became Read-Only: Preserve the First Error

When a Linux filesystem unexpectedly becomes read-only, preserve the earliest relevant storage and filesystem errors before attempting recovery. The read-only state may be a protective response to an earlier fault. Forcing writes back on can hide the sequence without correcting the cause.1

First establish what actually failed: a mount marked read-only, a filesystem that has shut down, an underlying device problem, or an application denied access for another reason. These can produce similar user-facing symptoms but require different investigations.

Read article
Hosting

Deleting Guest Files Did Not Shrink the Datastore: Trace Discard Support

Deleting files inside a virtual machine frees space for that guest filesystem to reuse. It does not prove that the virtual disk, datastore and storage array have released the corresponding allocation. Reclamation depends on the discard path and the provisioning rules at each layer.1, 3

Start by naming the measurement that has not changed. Guest free space, allocated VMDK space, datastore free space and array physical consumption are different observations. A successful operation at one layer is not a receipt from every layer below it.

Read article
Performance

.NET Requests Stall with Low CPU: Diagnose Thread Pool Starvation

A .NET service can respond slowly while using little CPU because its ThreadPool workers are blocked rather than computing. Diagnose starvation by correlating request latency, worker-count changes, queued work and completed work, then identifying what occupies the workers. Low CPU alone is not proof.1

The practical consequence is that buying more CPU or increasing concurrency may leave the blocking operation intact. The useful question is which work cannot obtain a worker promptly, and why existing workers cannot return to the pool.

Read article
Performance

.NET Process Memory Keeps Rising: Prove Retention Before Calling It a Leak

Rising process memory is a reason to investigate, not sufficient proof of a managed-memory leak. Separate process memory, managed-heap capacity, allocation activity and objects that remain reachable. Then identify the owner keeping unexpected objects alive.1, 4

A useful diagnosis explains both the growth and the unwanted lifetime. “There are many strings” describes a heap; “a long-lived collection retains completed requests without eviction” identifies a candidate defect that can be tested.

Read article
Troubleshooting

SQL Connection Pool Timeout: Find Leaked or Long-Held Connections

A connection-pool timeout means the application could not obtain a usable connection within the allowed wait. It does not prove that SQL Server refused a login, and it does not prove a leak. Connections may be undisposed, held by slow work, reserved by transactions or divided among unexpectedly many pools.1

Start with the exact exception and the code operation that raised it. The durable correction follows the resource ownership and wait, rather than immediately increasing the pool limit.

Read article
Troubleshooting

ASP.NET Core Will Not Start on IIS: Diagnose 500.30 and 500.31

For an ASP.NET Core application that will not start on IIS, use the complete error code to choose the first investigation. A 500.30 indicates an in-process startup failure; a 500.31 identifies failure to find native dependencies, commonly a missing required runtime. The application event and startup exception must establish the actual cause.1

Do not start by reinstalling everything or repeatedly recycling the application pool. Preserve the failing deployment and its first useful startup evidence so a correction can be tied to a specific mismatch or exception.

Read article
Performance

Outbound HTTP Calls Fail Under Load: Check HttpClient Connection Lifetime

When outbound HTTP calls fail under load, inspect how the application owns its clients, handlers and connections. Creating and disposing a new owned handler for every request can cause connection churn and port pressure. Keeping connections indefinitely can delay adoption of changed DNS answers.1

The correction is a deliberate lifetime strategy supported by measurements. Microsoft documents two common approaches: long-lived clients with a suitable pooled-connection lifetime, or short-lived clients created by IHttpClientFactory, which manages reusable handlers.1, 2

Read article
Performance

PHP-FPM Has Reached pm.max_children: Measure Queueing Before Adding Workers

A PHP-FPM warning that pm.max_children has been reached means the configured process limit was encountered. It does not, by itself, prove that adding workers is safe or that the pool is still saturated. Check current queueing, active workers, request duration and available resources before changing capacity.1, 2

The key question is why workers remain occupied. Legitimate concurrency, slow application code, database waits and external dependencies can all produce pressure at the same FPM limit. More workers may help one case while increasing contention in another.

Read article
Troubleshooting

RabbitMQ Queue Grows but Consumers Are Connected: Inspect Unacknowledged Work

A connected RabbitMQ consumer is not proof that messages are completing. Separate messages ready for delivery from deliveries awaiting acknowledgement, then compare delivery and acknowledgement rates with the consumer's own processing evidence.1, 3

A growing unacknowledged population can mean slow processing, blocked dependencies, excessive work in flight or a missing acknowledgement path. A growing ready population can mean arrivals exceed useful completion capacity. The shape of the backlog narrows the investigation; it does not identify the cause alone.

Read article
VMware

vCenter Sign-In Fails After a Certificate Expires: Find the Store Responsible

Direct answer: If the vSphere Client spins at sign-in, shows "no healthy upstream", "503 Service Unavailable" or "Signing certificate is not valid", or vCenter services such as vpxd will not start after a reboot, an expired certificate is one of the first things to check. vCenter uses several kinds of certificate, and they fail in different ways. The Security Token Service (STS) signing certificate is the one used to issue sign-in tokens. It is not covered by the general certificate expiry alarm. The Machine SSL certificate and the solution-user certificates live in the VMware Endpoint Certificate Store (VECS). Find which one expired, with read-only listings, before you change anything.

Read article
Troubleshooting

ESXi Host Not Responding or Disconnected in vCenter: Narrow the Cause

Direct answer: "Not Responding" means vCenter has stopped receiving the host's heartbeats. Broadcom's KB 323612 states that vCenter expects a heartbeat every 10 seconds. If it misses six in a row (60 seconds), it marks the host. The heartbeat travels from the host's vpxa agent to vCenter on UDP 902. The common causes are a blocked or lost UDP 902 path, a wrong managed IP address, a hung hostd or vpxa, or a failed host. "Disconnected" is different. vCenter has stopped managing the host, either because someone disconnected it, a reconnect attempt failed, or the host licence expired. A disconnected host stays disconnected until someone reconnects it (KB 337333).

Read article
VMware

vCenter appliance partition full: finding what filled /storage/log, /storage/db or /storage/seat

Direct answer: a vCenter Server Appliance keeps its logs, its database and its events/tasks/statistics data on separate logical volumes, each on its own virtual disk. When one fills, the symptoms depend on which one: a full /storage/log produces disk-exhaustion alarms, slow sign-in and sometimes 503 errors; a full /storage/seat stops the database from extending its files, so vCenter services can fail to start with "no healthy upstream". Start by confirming which mount point is full with df -h and df -i, then find the largest files or file counts inside that one mount point, then match what you find to a known Broadcom cause.

Read article
VMware

vMotion fails: telling CPU compatibility, device dependencies and the vMotion network apart

vMotion failures can involve CPU compatibility, inaccessible device backing or the vMotion network. Start with the migration wizard's compatibility result and the exact error, then investigate the layer it identifies.

This guide separates those causes and explains network checks, including the configured MTU. Changing EVC, disconnecting devices or altering the network can affect running workloads. Classify the failure before deciding which change is justified.

Read article
VMware

vSphere HA did not restart some VMs: reading the event trail and checking admission control and isolation settings

Direct answer: work VM by VM. For each VM that stayed down or was powered off unexpectedly, find the cluster event that explains HA's decision, then confirm it in /var/log/fdm.log on the primary host. The common reasons are all recorded. A VM override set restart priority to Disabled. Host Monitoring was off. HA found no compatible host, for example because of local storage or an ISO on a missing datastore. Failover capacity ran out. Or the host was judged isolated and its isolation response powered VMs off. Only after the per-VM reasons are known should you change policy.

Read article
VMware

AD Users Cannot Sign In to vCenter After a Domain Controller Certificate or Server Change

Direct answer: if AD users suddenly get "invalid credentials" at the vCenter sign-in page while the local SSO administrator still signs in, and the identity source is Active Directory over LDAP using LDAPS, the usual cause is that vCenter no longer trusts the certificate the domain controller now presents. LDAPS identity sources hold copies of the certificates you uploaded. When a domain controller renews its certificate, is replaced, or a load balancer in front of the domain controllers changes certificate, and what vCenter holds was the old leaf certificate rather than the issuing CA chain, the TLS handshake fails and vCenter reports a credential error.

Read article
VMware

Adding a vTPM to a VMware VM: Key Provider Requirements and What Changes Afterwards

Direct answer: a virtual TPM (vTPM) is a small device to add but a lasting change to the virtual machine. vSphere protects the vTPM's data with virtual machine encryption, so it needs a working key provider first. Adding the vTPM encrypts the VM home files (including the .nvram file that holds the vTPM state), although not the virtual disks. From then on the VM depends on that key provider. Power-on, console access, cloning, migration to another vCenter and restore all need the right keys and the right cryptographic privileges.

Read article
VMware

Planning a VM Hardware Version Upgrade in vSphere: When, How and What to Check

Direct answer: upgrade a VM's hardware version only when you need something the newer version provides. That might be a guest operating system that requires it, a device or limit it introduces, or CPU features needed to mitigate a security vulnerability. In practice it cannot be undone in place: ESXi has no downgrade, and the VM can no longer run on, or be vMotioned to, any host that does not support the new version. Before upgrading, confirm every host and vCenter that might run the VM (including DR and restore targets) supports the target version.

Read article
Troubleshooting

ESXi Purple Screen (PSOD): Preserve the Evidence and Collect the Core Dump

Direct answer: a purple diagnostic screen (PSOD) means the VMkernel, the core of ESXi, hit an error it could not recover from. The host has stopped. vSphere HA may restart protected VMs on other hosts, if restart is enabled for them and compatible hosts with access to their storage and enough capacity are available; check that the VMs and their applications actually came back. Before anyone reboots the host, photograph the whole screen: the error message, the CPU and world line, the stack trace, the uptime and the "Starting coredump" line. Then check that a core dump was actually written.

Read article
VMware

Planning a vSphere Major Upgrade: Order, Interoperability and a Go/No-Go Record

A major vSphere upgrade affects vCenter, ESXi and the products that depend on them. Check support dates, the supported upgrade path and Broadcom's product interoperability matrix before scheduling the work.

This guide covers the published update sequence, integration dependencies, removed features and pre-checks. Build a go/no-go record with a tested vCenter backup and rollback plan. Patching within an existing release is a separate operation.

Read article
Troubleshooting

Small Requests Work but Large Transfers Hang: Proving a Path MTU Black Hole

Direct answer: if ping and small requests across a link work, but file copies, large web pages, TLS handshakes with big certificate chains or database result sets stall and then time out, test for a Path MTU (PMTU) black hole before you suspect the application.

A black hole happens when a hop on the path cannot carry full-size packets, and the ICMP message that would tell the sender to send smaller packets never reaches it. With IPv4 that message is "Fragmentation Needed". With IPv6 it is "Packet Too Big". The message can be lost because a router does not send it or because a firewall filters it.

Read article
Troubleshooting

Port Forward Works From the Internet but Not Inside: NAT Hairpin or Split DNS

Direct answer: when a service published with a port forward works from the internet but fails for clients on the same internal network that use its public name, the usual cause is that the edge device does not hairpin. There are two ways it goes wrong:

Prove which case you have with a capture on the server and on the client. Then pick one of three fixes:

Read article
Troubleshooting

Connections Half-Work Through Two Gateways: Proving Asymmetric Routing

Direct answer: when a server or network has more than one way out, a request can arrive by one path and the reply leave by another. The usual cases are:

Stateful devices track connections, so the firewall that saw only the reply has no connection for it and drops it. The firewall that saw only the request never sees the handshake complete and times it out. A Linux host doing strict reverse-path filtering can drop packets that arrive on an interface it would not use to reach their source.

Read article
Performance

Proving Packet Loss and Latency to a Provider: Reading mtr and pathping Correctly

Direct answer: the most useful number in an mtr or pathping report is the loss and latency at the destination, and at the hops just before it. A loss percentage shown at one intermediate router, which does not carry through to the hops after it, is usually that router limiting or deprioritising the ICMP replies it generates itself. It is not loss of your traffic.

Loss that starts at some hop and persists through every later hop to the destination is a useful clue to real path loss, but not proof by itself, and not the only shape real loss can take.

Read article
Troubleshooting

Duplicate IP or MAC Address After a VM Clone or Restore: Prove It, Contain It, Fix the Copy

Direct answer: When users see intermittent drops, RDP sessions freezing or pings that work one minute and fail the next shortly after someone cloned, restored or test-recovered a virtual machine, suspect two machines claiming one network identity. There are two different faults. A duplicate IP address means the copy came up with the same static address as the original. A duplicate MAC address means two virtual NICs share one hardware address, so the physical switches keep moving that address between ports. Prove which one you have from evidence (Windows Tcpip event 4199, neighbour tables that change MAC, the vCenter MAC conflict alarm, a PowerCLI MAC listing).

Read article
Troubleshooting

Account Keeps Locking Out: Find the Source From Event 4740 to the Process

Direct answer: An account that locks out again minutes after it is unlocked is most often being fed an old password by something automatic: a service, a scheduled task, a mapped drive, a saved credential, a phone or mail client, or a disconnected remote desktop session still running as that user. Find it in three hops:

Raising or disabling the lockout threshold is not a fix. It hides the source, and a later password change will lock the account again.

Read article
Troubleshooting

Before Restricting NTLM: Inventory Who Uses It and Why Kerberos Was Not Used

Direct answer: Do not start with a "deny" policy. First turn on the three "Network security: Restrict NTLM" audit policies (domain controllers, servers receiving NTLM, and clients sending it), collect several weeks of the Microsoft-Windows-NTLM/Operational log plus Security event 4624 entries whose authentication package is NTLM, and turn each caller into a row: who, from where, to which target, and why Kerberos was not used. Most rows trace to a literal IP address, a missing or duplicate service principal name (SPN), hard-coded NTLM in an application, a local account, or a client with no line of sight to a domain controller.

Read article
Hosting

Windows LAPS: Deploy It Safely and Recover the Local Administrator After a Restore

Windows LAPS gives each managed server a separate local administrator password and controls who can retrieve it. This guide covers deployment prerequisites, delegated access, encryption and password history.

Learn what happens after restoring an older backup, when retained password history can help, and why a separate recovery plan is still needed if the restored password is no longer available.

Read article
Troubleshooting

Entra Connect Sync Errors: Duplicate Attributes, Soft and Hard Match Conflicts

Direct answer: When a user is missing from Microsoft 365, or a change made in Active Directory never reaches the cloud, the sync server almost always recorded why. Look at the export errors for the Microsoft Entra connector, name the error type, and find the other object involved: most failures are two objects claiming the same unique value (a UPN, a proxy address or a security identifier), or an on-premises object trying to take over a cloud object that the service now refuses to hand over. Fix the value in the directory the object comes from, usually on-premises AD, then let the next delta cycle carry it.

Read article
Troubleshooting

RD Gateway Connection Failures: Certificate, CAP, RAP and Transport Checks

Direct answer: If users can connect to a session host directly from inside the network but fail through RD Gateway, the fault is almost always in one of four layers, checked in this order:

For slow or unstable sessions through the gateway, rather than refusals, the transport is one possibility to check: if UDP 3391 is blocked or mishandled, the session runs over HTTPS only, which may contribute. Network loss, latency and session-host load can look the same, so confirm which one applies before you change anything. The Gateway operational log tells you which layer it is in nearly every case. Read it before changing policies or certificates.

Read article
Troubleshooting

"The Trust Relationship Failed" After a Member Server Restore: Prove the Secure Channel and Repair It in Place

A restored domain-member server may hold an older computer account password than Active Directory, causing a trust relationship failure. Confirm the secure-channel problem with read-only evidence and correlate it with the computer object on the domain controller.

This guide explains in-place repair and the credentials it requires. Removing and rejoining the server is not always necessary; deleting and recreating its computer object can lose group memberships and delegated permissions.

Read article
Hosting

Revoke a Departed Administrator's Access Across a Hosted Estate

Disabling an account does not remove every route into a hosted estate. Administrator access can also survive through cloud sessions, vCenter permissions, local accounts, SSH keys and shared credentials.

This checklist explains how to inventory those routes, revoke access in a controlled order, rotate shared secrets and verify the result. It also covers urgent containment and why previously issued access tokens may remain usable.

Read article
Troubleshooting

Clean Up Stale AD DNS Records: Scavenging Without Deleting Live Ones

Removing stale dynamic DNS records requires both zone aging and server scavenging to be configured. Their scope differs: aging replicates with the zone, while server scavenging settings do not.

Inventory records and explain old timestamps before enabling cleanup. Static records with zero timestamps are not scavenged and need an owner's review. This guide covers protection periods, a controlled rollout and checks that approved deletions disappear while records that must remain are preserved.

Read article
Hosting

Windows DHCP Failover: Partner Down, Lease Conflicts and Recovery

Windows DHCP failover partners exchange lease information over TCP port 647. Losing contact puts them into communication interrupted, where they continue serving clients within limits rather than immediately taking over the whole pool.

This guide explains partner-down decisions, lease conflicts, scope statistics and the maximum client lead time. Check the actual failover state and available addresses before recovery; a communications failure alone does not establish that the other server has stopped serving clients.

Read article
Cloud

SFTP Host Key Changed: Verify It and Rotate Keys Safely

An SFTP host-key warning can follow a legitimate server rebuild or key rotation, but it can also indicate an intercepted connection. Verify the change before accepting the new key; disabling host-key checking is not a repair.

For planned rotation, introduce the new key while the old one still works, let clients learn it, then retire the old key. This guide also covers changing the transfer account's key without breaking scheduled jobs.

Read article
Cloud

S3 Credentials for Automated Delivery: Least Privilege and Rotation

Direct answer: A delivery job that only writes files to one place should hold a credential that can only write to that place.

The AWS IAM User Guide says access keys "are long-term credentials". "IAM users with access keys are an account security risk". The recommended practice is to use temporary credentials, such as IAM roles, instead of creating long-term access keys. Temporary credentials expire after a short period; long-term keys "remain valid until you manually revoke them".

Read article
Backup & Recovery

SQL Point-in-Time Restore: Plan and Verify the Backup Chain

For native SQL Server point-in-time recovery, establish a valid full/differential/log sequence, the required keys and the exact transaction boundary before restoring to an isolated destination. This guide provides inspection and planning checks, not an executable production restore runbook. A readable backup is not proof of a recovered application.

The reference environment is SQL Server 2022. The procedure is source-researched and untested; rehearse it with synthetic data in an isolated lab first. No database was restored for this article, and no NetLeverage point-in-time recovery capability is implied.

Read article
Backup & Recovery

Restore a TDE Database Without the Original SQL Server

A certificate-protected TDE backup needs the original protecting certificate and its private key on the recovery instance. Record the protector thumbprint while the source is available, preserve the exports and their password through separate custody, and prove recovery on an independent instance. A certificate name or public certificate alone is insufficient.

This article uses SQL Server 2022 as its reference environment. It is a source-researched, untested recovery-design procedure. No certificate, key or database was exported or restored. The principal workflow concerns certificate-protected TDE; external key management requires its own provider-specific recovery design.

Read article
Troubleshooting

SQL Transaction Log Keeps Growing: Find the Real Cause

For a full or rapidly growing SQL transaction log, first measure used log space, the reuse wait, recent log backups, open transactions and the file/disk limits. Repair the dependency identified by that evidence; capacity relief may be needed while it is repaired. Do not delete the LDF, switch recovery model or schedule shrink as a shortcut.

This guide uses SQL Server 2022 as its reference. It is source-researched and untested. The diagnostic query reads existing metadata; no log was shrunk, transaction terminated or recovery model changed for this article. Rehearse collection and any subsequent remedy in an approved lab.

Read article
Troubleshooting

The Database Restored, but the Application Cannot Log In

A restored database can be online while its users cannot map to the intended destination logins. Confirm the actual server, error and identity before changing access. Compare SQL-user SIDs and, after an AD-domain change, Windows identities separately; an empty SQL-orphan query does not prove application access is correct.

This guide uses SQL Server 2022 on Windows as its reference. Its query is read-only, source-researched and untested. No accounts or permissions were changed. It focuses on SQL identity mapping after recovery; domain authentication and Kerberos require the separate companion investigation.

Read article
Backup & Recovery

SQL Server Reports Corruption: Preserve Evidence First

Preserve the complete SQL consistency errors and storage evidence, protect the available recovery sets, and assess a known-good isolated restore before considering repair. Do not run Windows disk repair while SQL Server is using the files. Non-repairing checks still need an agreed workload and permission scope.

This article uses SQL Server 2022 as its reference and is source-researched, untested guidance. It contains no executable repair command. Integrity checks, restores and any production containment action require an authorised DBA, resource planning and an isolated rehearsal where practicable.

Read article
Troubleshooting

SQL Kerberos After Migration: Check SPNs and NTLM Use

After a SQL Server move, test the actual remote connection’s authentication scheme, then compare its requested SPN with the current service account. Kerberos is attempted first; if a token cannot be obtained, Windows can use NTLM S1. A successful login can therefore conceal a broken Kerberos path, particularly where delegation is required.

This guide uses domain-joined Windows and SQL Server 2022 with a remote TCP connection as its reference. It is source-researched and untested. No directory object, SPN, service identity or authentication policy was changed. The diagnostic SQL example requires approved access and should be rehearsed in a lab.

Read article
Troubleshooting

IIS 502 and 503 Errors: Find the Failing Request Layer

For IIS 502 or 503, identify the responding layer, full substatus and matching request before restarting anything. A 503 can mean an unavailable pool or a queue limit; a 502 requires the module and ARR version to interpret it. If the request is absent from the site log, inspect HTTPERR as well as upstream logs.

This is source-researched, untested guidance for an IIS 10 reference environment. ARR-specific steps apply only where ARR is actually installed. Some Microsoft diagnostic examples describe older releases; their example timeout values and platform details are not assumed to be current configuration defaults.

Read article
Hosting

Renew IIS TLS Certificates and Verify the Live Endpoint

An IIS certificate renewal is complete only when the intended client endpoint presents the replacement certificate with a trusted chain and correct hostname. Check TLS termination, the actual HTTP.sys binding and intermediate certificates, then verify from the client path before retiring the old material.

This article uses an IIS 10 application as its reference. It is source-researched and untested; no certificate or binding was changed. Microsoft’s older IIS SSL guides provide useful mechanics, but their legacy protocol settings and sample troubleshooting fixes are not a modern security baseline.

Read article
Troubleshooting

DNS Changed but Users Still Reach the Old Application

When DNS has changed but an application still reaches the old server, compare authoritative answers, the client’s chosen resolver, its local cache and the application’s actual connection. Include negative caching and the internal DNS server’s cache. A correct lookup alone cannot prove that an established application session moved.

This guide concerns Windows DNS clients and servers, with Azure DNS documentation used only for its explanation of records and TTL. It does not deploy Azure resolvers or alter VPN routing. All procedures are source-researched and untested; no DNS record, cache or application was changed.

Read article
Troubleshooting

Windows Disk Full, VMware Space Free: Map the Layers

Free VMware datastore space does not make a full Windows volume larger. Measure the volume, partition, virtual disk and backing storage separately, then identify the writer consuming the space. Expand only after the correct layer, supported partition layout and operational headroom are established.

This article uses a Windows Server 2022 guest with a basic NTFS or ReFS data volume as its reference. It is source-researched and untested. No virtual disk, partition or filesystem was changed. Dynamic/spanned layouts, clustered disks and specialised storage require their own supported design.

Read article
Hosting

Scheduled Jobs After Migration: Prevent Duplicate Work

Before a migrated copy first boots, the migration lead must decide whether schedulers may start and isolate it from production destinations. Then inventory jobs, reconcile completed work and authorise exactly one executor per business operation. A successful interactive application test does not prevent duplicate exports, emails or payments.

This article covers Windows Task Scheduler and SQL Server Agent in a SQL Server 2022 reference environment. It is source-researched and untested. No task, job or server was changed. Application-native schedulers and external automation require the same ownership review through their own supported interfaces.

Read article
Backup & Recovery

Verify a Backup Delivered to S3 or SFTP Before Recovery

Before relying on a delivered backup, confirm its recovery-point identity, exact object/file, completeness and integrity, then rehearse recovery on an isolated destination. Test a ZIP archive’s members even when no producer hash exists, but do not confuse a valid archive with proof that the correct data or recovery point was supplied.

NetLeverage’s current Backup & Recovery page describes delivery or synchronisation to customer-controlled S3 or SFTP destinations and zipped copies. That published description supplies context only. This article does not claim that the service automatically produces a checksum manifest or performs the tests below.

Read article
Backup & Recovery

Size Backup Retention Using Measured Storage Growth

Size each backup destination from measured retained bytes and net growth, including versions and incomplete uploads, then reserve recovery space and provisioning lead time. Keep provider-run repository capacity separate from customer-controlled delivery storage. A retention count or protected-server size alone is insufficient.

This article is a source-researched, untested planning method, not a validated sizing calculator. No customer storage was inspected and no retention policy was changed. Amazon S3 provides the concrete versioning and lifecycle examples; other services and backup products require their own documented accounting.

Read article
Troubleshooting

Clock Drift in VMware-Hosted Windows: Trace the Source

Measure the Windows guest’s offset against the intended domain references, then correlate any jump with Windows Time policy and VMware Tools lifecycle corrections. Turning off periodic host synchronisation does not prove that one-off corrections are disabled. A snapshot event is evidence to investigate, not proof that every snapshot reset the clock.

The reference guest is domain-joined Windows Server 2022. The VMware source explicitly lists ESXi 7.x and 8.x; do not extrapolate its UI or settings to another release without checking that release. This procedure is source-researched and untested. No clock, service or time policy was changed.

Read article
Hosting

Server Up, Application Unusable: Build Better Health Checks

A useful health check must measure the operation users need, with separate signals for process liveness and readiness. In ASP.NET Core 8, both Healthy and Degraded return HTTP 200 by default; Unhealthy returns 503 S2. A monitor that accepts any 200 can therefore miss a declared degraded dependency.

This article is source-researched design guidance, not an implemented monitoring system. ASP.NET Core 8 provides a concrete reference for health-check concepts, but no application, probe or infrastructure configuration was changed or tested.

Read article
Backup & Recovery

VSS Writer Failed? Diagnose It Before the Next Backup

Direct answer: when a backup or quiesced snapshot of a Windows VM fails at the shadow copy stage, start inside the guest. Run vssadmin list writers from an elevated prompt and record every writer whose state is not [1] Stable or whose last error is not No error. For each one, read three things together: the phase in which it failed, its last error, and the time of the failed job. Then find the service or application that hosts it and read the Application log at that time. Decide whether that writer matters for this backup, and give the fix to the writer's owner.

Read article
Backup & Recovery

Linux VM Backups: Filesystem Freeze and Thaw Scripts

Direct answer: a quiesced VMware snapshot of a Linux VM asks VMware Tools to freeze the guest filesystems. That gives a filesystem-consistent copy: the filesystem's own transactions are complete and its data is flushed. It says nothing about the application's own transactions. A database restored from that copy may still recover correctly through its documented crash recovery, but only if everything it needs (for example data files plus write-ahead log, on every volume involved) was captured together.

Read article
VMware

VMware Tools Not Running or Out of Date: What It Breaks

Direct answer: VMware Tools (or open-vm-tools on Linux) is the in-guest component that reports guest information, carries out graceful shutdown and restart requests, and quiesces filesystems for snapshots. When it is not running, guest shutdown requests fail, quiesced snapshots can fail, and guest details in vCenter can be missing or stale. "Out of date" is several different statuses, from a routine newer version being available to a build known to be too old or to have a grave bug.

Read article
Backup & Recovery

Domain Controllers on VMware: Snapshots and Restores

Direct answer: a domain controller (DC) put back to an earlier disk state without Active Directory knowing can suffer a USN rollback: its later changes stop replicating, and replication tools can still report no error. Since Windows Server 2012, a DC on a hypervisor that provides VM-GenerationID (vSphere 6.x and later, per Broadcom) can detect a snapshot revert or a copy and apply safeguards: a discarded RID pool and a new invocation ID. Those safeguards limit damage. They do not bring back un-replicated changes, and Microsoft states they do not replace system state backups and the AD DS Recycle Bin.

Read article
Troubleshooting

Group Policy Not Applying: gpresult, Events and SYSVOL

Direct answer: When a Group Policy setting is missing on a server, decide first whether it is scope (the policy does not apply to that computer or user, including through loopback), processing (a run failed on network, authentication or resources) or replication (the domain controller used holds a different SYSVOL copy). Read the resultant set of policy for the affected user on the affected server, filter the Group Policy operational log to that user's run, then read SYSVOL state on the domain controllers. SYSVOL recovery methods are separate conditional procedures, not a ladder. Microsoft says of reinitialisation: "It's unnecessary in most cases, and it may cause data loss if done incorrectly."

Read article
Troubleshooting

Firewall Rules Stopped After a Move? Check the Profile

Direct answer: Windows Firewall rules are enabled per profile: Domain, Private and Public. Windows picks the profile per network connection. A domain-joined server gets the Domain profile on a connection only when it can find and authenticate to a domain controller through it; Microsoft states a new network is Public by default. A rule enabled only for the Domain profile then does not apply, and services that worked before a migration, restore, NIC replacement or re-addressing can be refused. Read each connection's category with Get-NetConnectionProfile, trace which domain-detection step fails (domain controller discovery, TCP 389, LDAP bind) and fix that dependency.

Read article
Troubleshooting

Static IP Lost After a VM Restore or Virtual NIC Change

Direct answer: When a VM is restored, converted, moved to a new hardware version or given a different virtual network adapter, the guest may see a new network device, and settings stored against the old device do not follow it. On Windows, the old adapter can remain as a hidden, non-present device that still holds the static address; Windows then warns that the address "is already assigned to another adapter" when you set it on the new adapter. On Linux, the interface can get a new name or MAC address, so configuration written for the old one no longer matches.

Read article
Troubleshooting

VM Will Not Boot After a Disk Controller Change: Diagnosis

Direct answer: A guest can only reach its boot disk if the driver for that disk's controller loads before the root file system is needed. If a VM's boot disk is moved to a controller type whose driver the guest does not load early, Windows stops with INACCESSIBLE_BOOT_DEVICE (Stop 7B) and a dracut-based Linux guest cannot find its root device. Prove which case you have with read-only checks first. Recovery and the switch are changes: follow the vendor procedure for your exact case with a verified backup, a rollback plan and a reviewer. This article gives no step-by-step recovery.

Read article
Troubleshooting

Linux VM in Emergency Mode After a Disk Change: fstab Checks

Direct answer: On a systemd-based Linux VM, local file systems listed in /etc/fstab become mount units that local-fs.target requires, unless the entry carries nofail, noauto or its own x-systemd.wanted-by= or x-systemd.required-by= dependencies. If one of those required entries names a device that no longer exists or no longer matches (a removed disk, a device name that moved, a file system re-created with a new UUID), the mount fails, local-fs.target is not reached and boot stops at the emergency shell. Find the failing entry from the installed system's own journal and findmnt --verify, compare its identifier with the disks that are present, and confirm the identifier is unique before any change.

Read article
Troubleshooting

IPsec VPN Tunnel Up but No Traffic: Read-Only Diagnosis

Direct answer: "Tunnel up" in a monitoring view can mean only that the IKE security association (SA) is established. Data crosses only if a child SA (CHILD_SA) exists for the exact source and destination networks, packets reach the gateway, they still match the IPsec policy after any NAT, the far gateway forwards them, and replies come back the same way. A child SA can fail on its own while the IKE SA stays up. On both gateways, read the installed child SAs, their traffic selectors and their counters, correlate the same SA and the same time window on each side, and use drop or error evidence before naming the failing layer.

Read article
Hosting

Remote Desktop Licensing Errors: Grace Period and CAL Mode

Remote Desktop licensing errors can involve an expired grace period, an unreachable licence server, the wrong licensing mode or incompatible client access licences (CALs). Reinstalling roles or buying licences may not address the cause.

Check the effective policy, run RD Licensing Diagnoser and compare CAL and host versions. This guide explains the 120-day grace period and the differences between Per User and Per Device licensing.

Read article
Troubleshooting

Print Spooler Crashes on Remote Desktop Hosts: Driver Checks

Direct answer: A Print Spooler stop on a Remote Desktop host has more than one candidate cause. Microsoft's printing troubleshooting guide names corrupted files in the spool folder and the spooler's service dependencies, and its driver documentation explains that a printer driver running inside the spooler process can halt the print service. Collect the crash evidence first and only treat a driver as the suspect when that evidence names it. Driver isolation contains a faulty driver by running it outside the spooler; it does not repair the driver. The repair is a fixed or replaced driver from the vendor, confirmed by a real print test.

Read article
Troubleshooting

Windows Cumulative Update Fails: DISM and SFC Diagnosis

Direct answer: When a cumulative update fails with 0x800f0831 (CBS_E_STORE_CORRUPTION) or 0x80073712 (component store inconsistent), Microsoft ties the error to the Windows component store, the set of packages and manifests servicing builds on. Other codes, such as 0x8024002E (update service disabled), point elsewhere, so read the code and CBS.log before anything else. Then run DISM's CheckHealth and ScanHealth and sfc /verifyonly: these scan without repairing, though they write to their logs. Only after that decide on a repair, using Windows Update or a repair source of the same OS version, language components and patch level. Never delete WinSxS content or force the update past its checks.

Read article
Troubleshooting

Windows Server Out of Memory With No Busy Process: Diagnosis

Direct answer: When a Windows server is short of memory and no process explains it, measure before naming a cause. Microsoft's first step is Performance Monitor: record Memory\Pool Nonpaged Bytes, Memory\Pool Paged Bytes and Paging File\% Usage over several hours. Only if pool keeps growing with a steady workload do you look for the pool tag that grows (PoolMon) and map it to a candidate driver; a handle count that keeps rising for one process is supporting evidence of the same kind of fault. A tag or count that rises is a lead, not proof: confirm it is not freed when the workload ends, then take it to the vendor.

Read article
Troubleshooting

Linux OOM Killer on a VM: Find the Cause, Set Memory Limits

Direct answer: When you suspect a Linux service was killed for lack of memory, find the record of the kill for the right boot before naming a killer: the kernel log entry (which, with the default oom_dump_tasks, includes a task table) or the systemd-oomd journal. A process can also exit on a failed allocation without either killer acting, so with no record the cause stays unresolved. Then establish which constraint applied: a cpuset, a memory policy, a memory (or swap) limit such as a cgroup memory.max, possibly on a parent slice, or the whole system.

Read article
VMware

How to automate VMware provisioning with Terraform and PowerCLI

Use Terraform to declare the VM's intended configuration and review its proposed changes. Use PowerCLI and an application check to verify the result. A successful apply confirms that the provider completed its operations; it does not prove that a useful service is running.

This guide provisions one disposable Linux VM from an existing template. It contains a complete small configuration, not a production landing zone. No Terraform initialisation, plan, apply or infrastructure connection was executed for this article. The reader must validate it against the installed CLI and approved lab before reuse.

Read article
Backup & Recovery

How to build and verify application-consistent VMware backups

Protect the VM and its application state, then recover a separate copy and prove a business operation. A successful backup job is one piece of evidence. It does not establish that the recovered database, service accounts, application dependencies and network configuration work together.

This guide uses Veeam Backup & Replication 13 as a concrete third-party reference workflow. It does not describe or imply a NetLeverage product capability. Confirm support for the exact backup-server, hypervisor, guest and application builds before using the procedure. All examples are for an authorised isolated lab; none has been executed for this article.

Read article
Cloud

How to build and validate a VMware private-cloud cluster

Build the cluster in stages: establish host and network consistency, configure resource policies, then prove migration and maintenance behaviour using disposable workloads. A cluster object is not evidence of availability. Its surviving hosts, storage paths and networks must support the workload after a failure.

This guide starts with installed ESXi hosts and a working vCenter appliance. Hypervisor installation, disk formatting, production power-loss tests and storage provisioning are excluded. Procedures are research-based lab instructions; they have not been executed for this article.

Read article
VMware

How to design and troubleshoot VMware networking

Start with an end-to-end traffic map, not a new distributed switch. A VMkernel adapter, virtual switch, physical uplink, switch port and destination interface must agree on the path being tested. A successful management ping does not prove that the vMotion or storage path works.

The examples use an isolated three-host lab. Configuration steps are unexecuted; production uplink migration and management cutover require an approved change window and tested console recovery.

Read article
Performance

How to find and fix ESXi CPU and memory bottlenecks

Diagnose a slow VM by combining application timings with guest and host measurements from the same interval. Adding vCPUs or memory without identifying the constrained layer can consume capacity while leaving the application unchanged.

This guide provides read-only collection and a controlled lab experiment design. VM resizing, reservations and host-wide power changes are separate configuration operations; no production tuning has been validated here.

Read article
Performance

How to diagnose VMware and vSAN storage latency

Map the slow application to its virtual disk and storage backend, then collect measurements at each boundary during the same workload. “The datastore is slow” is not a root cause: guest queueing, hypervisor queueing, physical devices, vSAN resynchronisation and network transport can all contribute.

This guide is diagnostic. It does not instruct you to delete snapshots, replace disks, change RAID layout or recreate disk groups. Those operations require a separate, validated recovery and maintenance procedure.

Read article
VMware

How to harden VMware without breaking operations or recovery

Harden one access boundary at a time and test both the permitted task and the prohibited task. A control is incomplete if it blocks an attacker but also removes the only working recovery path. This guide builds a staged lab procedure with explicit evidence for identity, access and recovery.

It is not a complete compliance benchmark or an automatic hardening script. No VMware configuration, identity provider or firewall was changed for this article. Production rollout needs the organisation's change review and a successful isolated rehearsal.

Read article
Backup & Recovery

How to implement and test VMware-to-Azure disaster recovery

Design the destination application, network and return path before enabling replication. Transferring changed disk blocks is only one part of recovery. Users also need identity, DNS, access routes, capacity and a known authoritative copy of their data.

This guide uses the modernised Azure Site Recovery workflow to recover VMware guests as Azure VMs. It is not Azure VMware Solution, a VMware-to-VMware replication guide or the older classical Site Recovery architecture. Procedures are unpublished, source-researched lab instructions; no replication or failover was executed for this article.

Read article
Performance

How to optimise SQL Server on VMware using waits and measured I/O

Begin with a slow SQL operation and determine whether it is spending time executing or waiting. A VM can have low average CPU while queries wait on locks, storage or memory grants. More vCPUs do not remove those waits automatically.

This guide uses read-only diagnostic SQL and an isolated tuning workflow. It does not apply arbitrary server configuration values or create indexes in production. Queries are source-checked examples, not lab-executed results.

Read article
Performance

How to optimise Windows Server and Remote Desktop on VMware

Optimise the operation users find slow, then use resource measurements to explain it. Windows startup, application launch, report execution, file access and RDP display response are different workloads. One synthetic disk score cannot establish that all of them became faster.

The procedures below are for an authorised lab VM. They have not been executed for this article. Administrator access is needed for the power-plan experiment and for some counter sets; read-only access is enough for part of the inventory.

Read article
Performance

How to benchmark VMware storage with DiskSpd without confusing caches and capacity

Use a storage load generator to answer a narrow question, then confirm the result with the application. A large IOPS number is not automatically a faster database or file service. Block size, concurrency, cache behaviour, durability and the storage path determine what that number means.

This guide supplies read-only workload examples for a pre-existing disposable test file. It never targets a raw device and contains no write benchmark. No load was run for this article. Even read-only load consumes shared capacity, so first use requires an authorised isolated lab and a bounded test window.

Read article
Hosting

How to build and test SQL Server availability groups on VMware

An availability group protects database availability through replicas; it does not automatically protect every dependency of the application. Build the cluster, database replication and client connection path as separate layers, then prove a committed application transaction survives a planned role change.

This guide starts with an existing validated two-node Windows Server Failover Cluster (WSFC), supported SQL installations and a dedicated synthetic database. It excludes SQL Failover Cluster Instances, shared guest disks, distributed availability groups and production cutovers. No SQL or cluster operation was executed for the article.

Read article
VMware

How to build and validate Windows Server templates for VMware

Build a template from a disposable reference machine, prepare its identity through a supported deployment method and test two separate clones before release. A VM that starts once is not a reusable image: the second deployment must also receive correct identity, networking, drivers and application state.

This guide's main path uses a manually generalised Windows Server 2022 Desktop Experience image. Automated VMware guest customisation is discussed as a separate path. No image or guest was changed for the article; first use requires review and an isolated lab rehearsal.

Read article
Cloud

How to connect VMware workloads to Azure with controlled routing and private DNS

Treat hybrid connectivity as three separate contracts: routes carry packets, DNS returns the intended addresses, and applications authenticate to the intended service. A connected VPN tunnel establishes none of those application outcomes by itself.

This guide uses a lab VMware network connected through a supported edge VPN device to an Azure hub VNet. It assumes approved Azure access and an operator responsible for the edge device. No cloud resources, firewall rules or DNS zones were changed for this article.

Read article
Hosting

How to deploy and validate FSLogix profiles for VMware-hosted RDS

Deploy profile containers as a storage and identity service, then prove that a user receives the correct persistent profile on another session host. Faster sign-in is useful only if settings persist, concurrent access is controlled and one user cannot read another user's profile.

This is a new-lab pilot for two RDS session hosts, an existing domain and an empty dedicated SMB share. Existing profile migration, concurrent multi-session access and production profile deletion are outside scope. No FSLogix installation or registry change was executed for this draft.

Read article
VMware

How to design and validate vSAN storage policies and maintenance headroom

Choose a storage policy from the failures the service must survive, then prove the objects can actually satisfy it. A policy name such as Gold or Fast does not explain protection, capacity, placement or the temporary space needed to change it.

This guide uses a healthy standard single-site vSAN lab cluster and a disposable VM. It does not remove disks, inject hardware failures or reduce production redundancy. Exact supported layouts must be checked for the installed architecture and update.

Read article
VMware

How to patch an ESXi cluster with vSphere Lifecycle Manager

Patch a cluster as a sequence of controlled host changes. First prove that workloads can leave one host, then update that host and verify its storage, networking and applications before proceeding. A compliant image is a software-state result, not proof of a healthy service.

This guide assumes an existing image-managed vSphere 8 lab cluster, compatible licensed features and working shared storage. It does not convert a baseline-managed cluster, upgrade a whole major release or supply an emergency bootbank rollback recipe. No host was remediated for this draft.

Read article
Backup & Recovery

How to back up and rehearse recovery of the vCenter appliance

Back up the vCenter appliance to storage that remains accessible when vCenter is unavailable, then prove that an operator can rebuild the management service. A repository containing files is not enough: the recovery workstation, installer, credentials, DNS, host access and encryption material must also be usable.

This guide is a rehearsal for a standalone lab vCenter and synthetic inventory. Enhanced Linked Mode, vCenter HA and production replacement require different coordinated procedures. No appliance was backed up, stopped or restored for this draft.

Read article
Performance

How to diagnose and tune Windows SMB file services on VMware

Start with the user's file operation and identify where its time is spent. A large sequential copy, opening thousands of small files and saving a document over SMB exercise different paths. Increasing network bandwidth helps only when the original operation is constrained by that bandwidth.

This guide uses two disposable Windows VMs and synthetic files on a dedicated share. It contains no measured customer results. Commands are read-only inspection examples; workload execution and configuration changes require an approved lab.

Read article
Troubleshooting

vCenter says "Virtual machine disks consolidation is needed": why, and scenario-specific recovery decisions to clear it

A virtual machine shows the warning "Virtual machine disks consolidation is needed". Snapshot Manager shows "Needs Consolidation/Delete Snapshots" (KB 316575). The VM usually still runs, but on a chain of extra disk files that nothing is merging.

This article explains how to investigate the holder and select a recovery path. A normal backup-proxy release and a removal-refused case need different decisions. Recovery work requires a demonstrated restore route and a case-specific plan; the explanations here do not establish a validated production runbook.

Read article
VMware

The -000001.vmdk and delta files in a VM folder: what each one is

You look in a VM folder and find files you did not create: names ending in -flat.vmdk, -000001.vmdk, -000001-delta.vmdk or -000001-sesparse.vmdk, plus .vmsd, .vmsn and -ctk.vmdk. Perhaps vCenter shows "Needs Consolidation", or a snapshot has vanished from Snapshot Manager, and you need to know which files matter.

This article explains each file, the two delta formats, and how to read the chain with file listings and named descriptor files. You need a shell on the ESXi host for steps 1 to 5; the datastore browser cannot show descriptor contents. Every command in "How to prove it" only reads.

Read article
Troubleshooting

A snapshot removal or consolidation task is stuck, or fails with a lock error

A snapshot removal or consolidation task can appear stuck or fail with a lock error. Record the task state and identify the lock holder before choosing a recovery procedure.

This guide covers evidence and scoped recovery decisions for VMFS, NFS and vSAN, including why an active consolidation must not be cancelled. The procedures have not been tested in your environment; validate the applicable vendor procedure in a lab before using it.

Read article
Troubleshooting

A VMDK says it is locked and I need to find out who holds the lock

A power-on fails with "Unable to access file ... since it is locked", a consolidation fails with a disk lock error, or a datastore file operation returns "Device or resource busy". Something holds the file, and the message does not say what.

A lock is a record that one host writes so other hosts leave a file alone. The read commands below report state; none is a release step. Every procedure here is untested from our side: run it in a lab first.

Read article
Troubleshooting

VM will not power on: "The parent virtual disk has been modified since the child was created"

A virtual machine refuses to power on, or a snapshot consolidation fails, and the error says a parent disk "has been modified since the child was created". In the VM log the same fault appears as "Content ID mismatch". A failed consolidation can show it as msg.disklib.CID_MISMATCH, which KB 301128 explains as the CID and parentCID not matching in the descriptors.

This article shows how to draw your chain, compare the state identifiers and judge which side of a broken pair changed, all without changing anything, and where diagnosis must stop for specialist recovery. A CID is not a checksum proving the disk contents are correct.

Read article
Troubleshooting

Incremental backups suddenly run as full after a CBT reset

Last night's backup was meant to be an incremental, and it read the whole disk. The job took far longer than usual and moved far more data. Your backup product may report that it could not use changed block tracking, or it may simply record a full backup.

The cause can be that the hypervisor's record of changes was lost or deliberately discarded. That record is Changed Block Tracking (CBT). A backup can no longer rely on an invalid saved change position and may need a new valid baseline; the product determines the supported data-read workflow. This article explains the evidence, limitations and acceptance checks for recovery of protection.

Read article
Troubleshooting

Changed Block Tracking was lost after a Storage vMotion, an HA restart or a power event

Your backup product reported that an incremental backup turned into a full one, or that it could not use its saved change position, and the only thing that happened to the virtual machine was a move, a failover or an unclean stop. Sometimes there is no error at all, but the incremental backups look the wrong size.

The sources reviewed here do not establish a universal CBT-reset rule for every Storage vMotion or HA restart. KB 320557 says power failure or hard shutdown might reset tracking. Investigate the event and the exact build/product behaviour; a known defect and an unexplained event are different cases.

Read article
Troubleshooting

Backup slow or "Cannot use mode hotadd": which VDDK transport mode ran

A backup finished but moved data slowly, or the job log says one transport mode could not be used and another was tried, or backups hang while the host logs complain about memory for "NFC". All three trace back to how the Virtual Disk Development Kit (VDDK) carried the disk data to the backup proxy.

This article explains how each mode moves data, what each needs, the log lines that show a rejection, and how to choose. The proof steps only read. One gap: no official page with SAN failure messages could be fetched, so SAN rejections have no quoted log text here.

Read article
Troubleshooting

A backup failed and a production disk is still attached to the backup proxy

You open vCenter and the production VM shows a "Virtual machine disks consolidation is needed" warning. Consolidation fails with "Failed to lock the file" or "One or more disks are busy". The backup proxy VM, the helper machine your backup product uses to read other VMs' disks, lists a hard disk you never gave it.

This article shows how to prove that, how to release the disk through scoped recovery decisions, and which order applies when vSphere refuses to remove the disk.

No recovery procedure here has been lab validated, including those described by vendor sources.

Read article
Troubleshooting

My VM freezes for a while when a snapshot is created or removed

A VM drops pings, a remote console session disconnects, or an application times out at the moment a snapshot is created or deleted, for example while your backup product runs. The VM comes back by itself, but the pause was long enough for users or monitoring to notice.

This article shows how to read stun lengths from the VM's log, what the cited Broadcom pages say makes them long, and how to assess changes against application and recovery requirements. The "How to prove it" section only reads.

Read article
Backup & Recovery

VMware Snapshots vs Backups: A Practical Guide to Recoverability

A successful backup-job status does not demonstrate that the business service can be restored. An actual restore, followed by application validation, provides evidence. The practical question is whether staff can resume essential work with acceptable data loss and downtime.

For a business owner or IT manager, this changes the conversation with a backup partner. Ask for recovery objectives, the failures covered and a record of restored business operations. A completed job is useful operational information; application acceptance establishes what the recovered service can actually do.

Read article