Kernel Data Recovery Blog

Exchange DAG in witness failed state – How to fix?

Read time 4 minutes

Microsoft Exchange Server promises a highly available and resilient framework in which the servers will be efficient enough to face the majority of challenges and provide access to the users. Database Availability Group is part of such a framework that has multiple mailbox servers (up to 16 servers) to host databases. The DAG runs an automatic database-level recovery with its failover mechanism.

Witness Server in Database Availability Group

When the DAG has an even number of members, a witness server is a server that serves outside the DAG that is used to establish and preserve quorum. A witness server is not used by DAGs that have an odd number of members. A witness server is required for all DAGs with an even number of members. Any PC running Windows Server can serve as the witness server. The witness server’s operating system does not have to be the same version of Windows Server as the DAG members’ operating system.

After updating the Windows Server and restarting the Exchange Server, you may get a warning or error while checking the status of the database availability group.

This is the cmdlet:

Get-DatabaseAvailabilityGroup -Identity <Database-Availability-Group-Name> -Status | ft Name, Witness*, Servers

As a result of the command, the warning message appears like the following:
WARNING: Database availability group <Database-Availability-Group-Name> witness is in a failed state. The database availability group requires the witness server to maintain quorum. Please use the Set-DatabaseAvailabilityGroup cmdlet to re-create the witness server and the directory.

The warning message explains the correct method to recreate the witness server using the Exchange Management Shell cmdlets.

NOTE – The cmdlet will work only in on-premises Exchange Server.

Set-DatabaseAvailabilityGroup -Identity <Database-Availability-Group-Name> -WitnessServer <New-Server-Name> -WitnessDirectory <NonRootLocalLongFullPath>

Now, run the first cmdlet, and it will run without any warning or error.

Get-DatabaseAvailabilityGroup -Identity <Database-Availability-Group-Name> -Status | ft Name, Witness*, Servers

You can change the Witness server in the Exchange Admin Center too.

  1. Select “Servers” > “Database Availability Groups” in EAC.
  2. After choosing the DAG, click the edit icon.
  3. Click Save after entering the new witness server’s FQDN and new witness directory path.

Check the server’s name under servers > database availability groups to confirm the DAG witness server. Additionally, make sure the witness server’s witness directory is correctly built.

There are some great practices to put the witness server in the data center when the single or multiple Database Availability Groups.

DAG in Datacenter Witness Server
A single Database Availability Group in a single data center. Keep the witness server in the same data center as the Group members.
A single Database Availability Group in two data centers.
  • Keep the witness server in the primary data center.
  • Keep the witness server in Microsoft Azure Virtual Network, as it will enable the automatic data center failover.
Multiple Database Availability Groups are deployed in a single data center. Keep the witness server in the same data center as the group members.
Multiple Database Availability Groups are deployed in two data centers. Keep the witness server in a data center that you consider the primary one for each group.
Single/multiple Database Availability Groups are deployed in more than two data centers. The witness server in this configuration needs to be in the data center where most of the quorum votes are expected to occur.

Conclusion

The DAG’s witness server is a crucial part that is necessary to keep a quorum in place. The failover clustering, however, can be broken if a witness server goes down or fails after a reboot, leaving it in a failed witness server state. You must either try to bring the witness server online in such a serious circumstance or switch to a different witness server and witness directory. You can utilize your Exchange Server backup to restore the database and mailboxes if the member server malfunctions during this procedure or the database is dismounted because of discrepancies.

If you have not taken the backup of the latest changes on the server, then the best option is to repair the affected database and recover the mailboxes. Kernel for Exchange Server, an advanced Exchange EDB Recovery keeps a database consistent by scanning it for corruption and saving it in a live Exchange Server. There are multiple destinations available for your data, and you can check the preview of recovered items after completing the scan.