Australian cloud for business software — built and run in Sydney since 2002

NetLeverage
NetLeverage

DFSR event 2213: replication stopped after a dirty shutdown, and how to resume it safely

Technical article
Troubleshooting

About this article: This article is reviewed against the cited sources. This is source research, not lab validation. Check the stated versions and your own requirements before making changes.

Event 2213 in the DFS Replication log means the service found its database was not shut down cleanly, and it is waiting for an administrator before it replicates that volume again. The event text includes a ready-made command to resume replication. Run it, but only after you have a backup of every replicated folder on the volume.

The pause is deliberate. Microsoft describes it as a safeguard that lets you capture changes made since the last backup before recovery runs, because dirty shutdown recovery can produce large backlogs, which can cause replication conflicts. The page says that in some cases, before the fix in hotfix 2780453 was released, the winning file might not be the version the user wanted.

What the error means

DFS Replication (DFSR) keeps one Extensible Storage Engine (ESE) database per volume that hosts a replicated folder. The database holds metadata about each file and folder. When the DFSR service is told to stop, it commits outstanding changes to that database. A dirty shutdown is the case where the service could not finish committing before it stopped. At the next start, DFSR checks the database's integrity.

Microsoft's page explains the history. After hotfix 2663685 (or a later Dfsrs.exe) on Windows Server 2008 R2, DFSR no longer recovered the database automatically after a dirty shutdown. It logged event 2213 and waited for an administrator to resume replication. The page states that Windows Server 2012 behaves this way by default. It also says that after hotfix 2780453 you no longer have to pause replication during a dirty shutdown, and that this fix is in all Windows Server 2012 default media.

Two points of scope matter. The page names Windows Server 2008 R2 SP1 and Windows Server 2012 and does not say how later releases behave. And it does not print the event's wording, only that the event text carries the resume command with the right volume GUID. Read the event on your own server rather than trusting a pasted copy.

Likely causes

Cause What you see How to confirm
Hard reboot or power loss Event 2213 after the unplanned restart; replication does not resume The System log shows the restart was not a clean shutdown; Microsoft names power outages and hard reboots
DFSR could not finish within the service shutdown allowance Events 2212 and 2214 on most restarts, or 2213 on every restart if AutoRecovery is enabled Compare the times of DFSR events 1006 (DFSR is stopping) and 1008 (DFSR stopped); Microsoft says a service has 30 seconds before the Service Control Manager forces it to close
Stop-replication behaviour still in force Replication waits for you after every dirty shutdown The StopReplicationOnAutoRecovery value under the DFSR Parameters key is absent or not 0
A busy hub server Long gap between events 1006 and 1008 Microsoft says a busy hub server may need more than 30 seconds to commit its changes

Prove the cause before changing anything

These checks only read.

  1. In Event Viewer, open the DFS Replication log. Find event 2213 and note its time and its text. The event text carries the resume command and the volume GUID for that server.
  2. Note whether events 2212 and 2214 appear around the same time. Microsoft pairs them with recovery.
  3. Look at the System log for the cause of the restart: a power loss, a crash, or a forced stop.
  4. List which replicated folders live on the volume named in the event. DFSR keeps one database per volume, so every replicated folder on that volume is affected.
  5. Open Registry Editor and look at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\DFSR\Parameters for a DWORD named StopReplicationOnAutoRecovery. If it is missing, Microsoft's text says you create it when you need it.
  6. For the shutdown-time cause, look at the times of events 1006 and 1008 across several restarts. Microsoft says one restart is not an adequate sample, because the service needs time to accumulate pending transactions.

Read the results this way: a single 2213 after a known power cut points to the first cause and needs one resume. A 2213 or a 2212 and 2214 pair on most restarts points to a server that needs longer to stop.

Fix

Resume replication after event 2213. This is the sequence Microsoft documents.

Disruptive step: resuming replication starts dirty shutdown recovery, which can produce a large backlog and conflict resolution. Microsoft says skipping the backup may result in data loss from unexpected conflict resolution, and there is no way to undo a conflict result after the fact; the backup is the way back.

  1. Back up the files in all replicated folders on the volume. Microsoft says skipping this can cause data loss from unexpected conflict resolution during recovery.
  2. From an elevated command prompt, run the ResumeReplication WMI method of the DfsrVolumeConfig class. Microsoft's example takes this form, with your own volume GUID:
wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="E18D8280-2379-11E2-A5A0-806E6F6E6963" call ResumeReplication

The GUID shown above belongs to Microsoft's example. Copy the command from the text of your own event 2213 instead. In PowerShell, Microsoft says to add single quotation marks around the where clause:

wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where 'volumeGuid="<your volume GUID>"' call ResumeReplication

The page's later example lines run path and dfsrVolumeConfig together as pathdfsrVolumeConfig. That reads as a typing slip; the command in your event text is the one to use.

  1. A successful run reports Method execution successful with ReturnValue = 0.
  2. Check that events 2212 and 2214 have been logged on that server.

If a replicated folder must be reinitialised after a dirty shutdown. Microsoft's order is to disable the replicated folder, run the resume steps above, then enable the replicated folder. If you disable and enable the folder before the resume command, initial synchronisation does not happen because the volume manager is offline.

Risk: disabling a replicated folder stops replication of that folder, and re-enabling it can discard private replication data. The 2213 page names no tool for the disable step. The Set-DfsrMembership page documents -DisableMembership and says DFSR keeps the folder and its private data, but that re-enabling means you lose private data and can lose replicated data. The stand-alone DFS Management route is not covered by either page. Preserved files may be recoverable with Restore-DfsrPreservedFiles. Back out by restoring from the backup you took first.

Stop DFSR waiting for you (optional). To have DFSR perform AutoRecovery on its own, set the registry value Microsoft names:

  • Key: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\DFSR\Parameters
  • Value: StopReplicationOnAutoRecovery
  • Type: DWORD
  • Data: 0

On Windows Server 2008 R2 install hotfix 2780453 first. Microsoft says the change can be deployed on all versions of Windows Server 2012 and that you create the value if it does not exist.

Risk: registry edit. Setting 0 means a future dirty shutdown recovers without waiting for you, so you lose the chance to back up first. Back out by deleting the value; the page documents no other value. Microsoft's page includes a table of best practice for AutoRecovery by role and release. For Windows Server 2012 it shows "On" for a domain controller, a cluster node, a writable DFSR server and a read-only DFSR server, and "Off" only for a writable DFSR server on Windows Server 2008 R2 without hotfix 2780453. The page does not say how "On" and "Off" map to the registry value, so read the table for your release before choosing.

Give DFSR longer to stop. Where the service needs more than 30 seconds, Microsoft documents the WaitToKillServiceTimeout string value under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control, in milliseconds. Its example is 300000, which is five minutes.

Risk: this value affects all services, not just DFSR, and a shutdown can take longer. Microsoft says to use the lowest value that gives DFSR enough time and that the maximum is one hour; above that, the Service Control Manager returns to 30 seconds. Install hotfix 2549760 on Windows Server 2008 R2 first. To back out, remove the value to return to the default.

Confirm the fix

  • Events 2212 and 2214 are logged after your resume command, as Microsoft directs you to check, and the server does not log a new 2213.
  • Replication catches up. Watch the backlog fall rather than judging by one quiet hour; Microsoft warns recovery can create a large backlog.
  • After the next planned restart, the DFS Replication log shows events 1006 and 1008 close together and no new 2213.

Example (illustrative): fs01 and fs02 replicate D:\Data for CONTOSO. A power cut at 02:10 stops fs01 uncleanly. On Wednesday morning event 2213 is in the DFS Replication log. The administrator backs up D:\Data on fs01, copies the command from the event and runs it, and then finds events 2212 and 2214 in the log. Later, restart tests show event 1006 at 02:00:10 and event 1008 at 02:01:45. That is 95 seconds (1 minute 35 seconds), well over the 30-second allowance. The administrator sets WaitToKillServiceTimeout to 120000, which is 120 seconds, leaving 25 seconds spare, and installs a UPS.

What to watch afterwards

  • Count 2212, 2213 and 2214 events per restart. A pair on every planned restart points to a shutdown allowance that is too short.
  • Keep a current backup of replicated folders. The safeguard exists so that you can capture changes before recovery.
  • Check that the server's power protection works. Microsoft recommends an uninterruptible power supply so DFSR servers can shut down gracefully.

Risks and limits

These commands have not been run or lab-tested. The Microsoft page is written for Windows Server 2008 R2 SP1 and 2012, and uses the wmic form; it does not describe other tooling or later releases. Do not delete the DFSR database or the DfsrPrivate folder to clear the event; Microsoft's page documents only the resume method and the disable and enable sequence above.

Sources