Quiesced Snapshots for Exchange 2010 on vSphere 5.0

Due to an upgrade from vSphere 4.0 to 5.0 some of our virtual machines stopped working in VEPA/vStorage backup sessions. I’ve noticed that a quiesced snapshot using vCenter was not possible. I was able to fix the most of them by reinstalling VMware Tools to fix some VSS inconsistencies, but an Exchange 2010 VM caused some additional issues.

vCenter reported the following warnings during creation of a quiesced snapshot.

Create virtual machine snapshot / An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.

The guest OS has reported an error during quiescing. The error code was: 5 The error message was: 'VssSyncStart' operation failed: IDispatch error #8449 (0x80042301)

Check the output of vssadmin list writers to determine the status of available VSS writers. In our case the Exchange writer reports an error.

C:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

[...]

Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {10552a00-36bf-401a-9141-4436d61f0d2c}
State: Failed
Last error: Retryable error

[...]

The event logs of the Windows 2008 R2 VM had the following errors, pointing to a VSS Provider related issue. Strange but true, the Exchange 2010 VSS Backups using Data Protector work as expected without any problems.

Log Name: System
Source: VDS Basic Provider
Event ID: 1
Task Category: None
Level: Error
Description:
Unexpected failure. Error code: D@01010004


Log Name: Application
Source: MSExchangeIS
Event ID: 9708
Task Category: Exchange VSS Writer
Level: Error
Description:
Exchange VSS Writer (instance 9a348c8f-7bdc-4dc6-a142-a5da187b36a1:2) failed with error code -2147467259 when checking volume dependencies for storage group '00000000-0000-0000-0000-000000000000' in preparation for backup.


Log Name: Application
Source: ESE
Event ID: 2007
Task Category: ShadowCopy
Level: Error
Description:
Information Store (2824) Shadow copy instance 2 aborted.


Log Name: Application
Source: MSExchangeIS
Event ID: 9609
Task Category: Exchange VSS Writer
Level: Error
Description:
Exchange VSS Writer (instance 9a348c8f-7bdc-4dc6-a142-a5da187b36a1:2) failed with error code 1006 when preparing for Snapshot.


Log Name: Application
Source: VSS
Event ID: 8229
Task Category: None
Level: Warning
Description:
A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried,
the error may not reoccur.
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

Operation:
PrepareForSnapshot Event

Context:
Execution Context: Writer
Writer Class Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Name: Microsoft Exchange Writer
Writer Instance Name: Exchange Information Store
Writer Instance ID: {184ee5a1-7dd2-42e0-aa74-a95ec549ed02}
Command Line: "C:\Program Files\Microsoft\Exchange Server\V14\bin\store.exe"
Process ID: 2824

After some research, I’ve found VMware KB article 1031298.

Cannot take a quiesced snapshot of Windows 2008 R2 virtual machine
http://kb.vmware.com/kb/1031298

The flag disk.EnableUUID needs to be changed to false on per VM basis. This flag controls the VSS creation mode which can be either application-level (true) or file-level quiescing (false).


With this change in place, the vStorage backup of the virtual machine works like a charm.
 

Leave a Reply