orchestrating Exchange with #vCO

Microsoft Exchange is a system that is ideally suited for automation.  It’s in almost every environment.  It has it’s own add-on to PowerShell that makes it easy to write scripts to handle tasks.  And most of the tasks that administrators perform after setup are rote tasks that are easily automated such as setting up mailboxes and adding IP addresses to a receive connector. 

Why vCenter Orchestrator?

Exchange already contains a robust automation platform with the PowerShell-based Exchange Management Shell.  This platform makes it easy to automate tasks through scripting.  But no matter how well these scripts are written, executing command line tasks can be error-prone if the end users of the scripts aren’t comfortable with a command line.  You may also want to limit input or provide a user-friendly interface to kicking off the script.

So what does that have to do with vCenter Orchestrator?  Orchestrator is an extensible workflow automation tool released by VMware and included with the vCenter Server license.   It supports Windows Remote Management and PowerShell through a plugin.

Start By Building a Jump Box/Scripting Server

Before we jump into configuring Orchestrator to talk to Exchange, we’ll need a Windows Server that we can configure to execute the scripts that Orchestrator will call.  This server should run Windows Server 2008 R2 at a minimum, and you should avoid Server 2012 R2 because the Exchange 2010 PowerShell cmdlets are not compatible with PowerShell 4.0. 

You will need to install the Exchange management tools on this server, and I would recommend a PowerShell IDE such as PowerGUI or Idera PowerShell Pro to aid in troubleshooting and testing.

Orchestrator and Exchange

As I mentioned above, Orchestrator can be used with PowerShell through a plugin.  This plugin uses WinRM to connect to a Windows Server instance to execute PowerShell commands and scripts.   In order to use this plugin, Orchestrator needs to be configured to support Kerberos authentication.

When I was testing out this combination, I was not able to get the Exchange Management Shell to load properly when using WinRM.  I think the issue has to do with Kerberos authentication and WinRM.

When you use WinRM, you’re remoting into another system using PowerShell.  In some ways, it is like Microsoft’s version of SSH – you’re logging into the system and working from a command line. 

The Exchange cmdlets add another hop in that process.  When you’re using the Exchange cmdlets, you’re executing those commands on one of your Exchange servers using a web service.  Unfortunately, Kerberos does not work well with multiple hops, so another to access the remote server is needed.

Another Option is Needed

So if WinRM and the Orchestrator PowerShell plugin don’t work, how can you manage Exchange with Orchestrator?  The answer is using the same remote access technology that is used for network hardware and Unix – SSH.

Since Exchange is Active Directory integrated, we’ll need an SSH server that runs on Windows, is compatible with PowerShell, and most importantly, supports Active Directory authentication.   There are a couple of options that fit here  such as the paid version of Bitvise, FreeSSHd, and nSoftware’s PowerShell Server.

There is one other catch, though.  Orchestrator has a built-in SSH plugin to support automating tasks over SSH.  However, this plugin does not support cached credentials, and it runs under whatever credentials the workflow is launched under.  One of the reasons that I initially looked at Orchestrator for managing Exchange was to be able to delegate certain tasks to the help desk without having to grant them additional rights on any systems. 

This leaves one option – PowerShell Server.  PowerShell Server has an Orchestrator Plugin that can use a shared credential that is stored in the workflow.  It is limited in some key ways, though, mainly that the plugin doesn’t process output from PowerShell.  Getting information out will require sending emails from PowerShell.

You will need to install PowerShell Server onto your scripting box and configure it for interactive sessions.

PowerShell Server Settings

Configuring the Exchange Management Shell for PowerShell Server

PowerShell Server supports the Exchange Management shell, but in a limited capacity.  The method that their support page recommends breaks a few cmdlets, and I ran into issues with the commands for configuring resource mailboxes and working with ActiveSync devices. 

One other method for launching the Exchange Management Shell from within your PowerShell SSH session is by using the following commands:

'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'
Connect-ExchangeServer –auto
If you try that, though, you will receive an error that the screen size could not be changed.  This is due to the commands that run when the Exchange Management Shell loads – it resizes the PowerShell console window and prints a lot of text on the screen. 
The screen size change is controlled by a function in the RemoteExchange.ps1 script.  This file is located in the Exchange Install Directory\v14\Bin.  You need to open this file and comment out line 34.  This line calls a function that widens the window when the Exchange Management shell is loaded.  Once you’ve commented out this line, you need to save the modified file with a new file name in the same folder as the original.
Edit RemoteExchangePS1
In order to use this in a PowerShell script with Orchestrator, you will need to add it to each script or into the PowerShell profile for the account that will be executing the script.  The example that I use in my workflows looks like this:

'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange-Modified.ps1'
Connect-ExchangeServer –auto

Note: It may be possible to use the method outlined by Derek Schauland in this TechRepublic article in place of modifying the EMS script.  However, I have not tested this with technique with Orchestrator.

Putting It All Together

Earlier this month, I talked about this topic on vBrownbag, and I demonstrated two examples of this code in action.  You can watch it here.

One of the examples that I demonstrated during that vBrownbag talk was an employee termination workflow.  I had a request for that workflow and the scripts that the workflow called, so I posted them out on my github site.  The Terminate-DeactivateEmail.ps1 script that is found in the github repository is a working example. 

Scripting Exchange 2010 Backups on Windows Server 2008R2 using PowerShell and Windows Backup Service

I’ve struggled with backing up my Exchange 2010 SP1 environment in my home lab since I upgraded over a month ago.  Before I had upgraded, I was using a script that did Volume Shadow Services (VSS) backups.

After upgrading, I wanted to cut my teeth with Windows Server Backup (WBS).  Windows Server Backup is the replacement for the NTBackup program that was included with Windows until Vista, and it uses VSS to take snapshot backups of entire volumes or file systems.

Unlike NTBackup, WBS will not run backup jobs to tape.  You will need to dedicate an entire volume or use a network folder to store your backups.  If you use the GUI, you can only retain one backup set, and a new backup will overwrite the old.

This was an issue for me.  Even though I have Exchange configured to retain deleted items for 14 days and deleted mailboxes for 30 days, I like to keep multiple backups.  It allows me to play with multiple recovery scenarios that I might face in the real world.

And that is where PowerShell comes in.  Server 2008R2 allows users to create a temporary backup policy and pass that policy to the Windows Backup Service.  This will also allow you to change the folder where the backup is saved each time, and you can easily add or remove volumes, LUNs, and databases without having to reconfigure your backup job each time.

I started by working from the script that Michael Smith that I linked to above.  To modify this script to work with WBS, I first had to modify it to work with Exchange 2010.  One of the major differences between Exchange 2007 and Exchange 2010 is that storage groups have been removed in the latter.  Logging and other storage group functions have been rolled into the database, making them self-contained.

The original script used the Get-StorageGroup PowerShell command to get the location of each storage group’s log files.  Since this command is no longer present, I had to add sections of this function to the function that retrieved the location of the database files.

After adding some error handling by using Try/Catch, the section that locates mailbox databases looks like:

foreach ($mdb in $colMB)
if ($mdb.Recovery)
write-host ("Skipping RECOVERY MDB " + $mdb.Name)
write-host ($mdb.Name + "`t " + $mdb.Guid)
write-host ("`t" + $mdb.EdbFilePath)
write-host " "

$pathPattern.($mdb.EdbFilePath) = $i

$vol = $mdb.EdbFilePath.ToString().SubString(0, 2)

#This Section gets the log file information for the backup
$prefix  = $mdb.LogFilePrefix
$logpath = $mdb.LogFolderPath.ToString()

## E00*.log
$pathpattern.(join-path $logpath ($prefix + "*.log")) = $i

$vol = $logpath.SubString(0, 2)

$i += 1
Write-Host "There are no Mailbox Databases on this server."

I also removed all of the functions related to building and calling the Disk Shadow and RoboCopy commands.  Since we will be using WBS, there is no need to manually trigger a VSS backup.

Once we know where our mailbox and public folder databases and their log files are located, we can start to build our temporary backup job.  The first thing we need to do is create a new backup job called $bpol by using the New-WBPolicy cmdlet.

##Create New Backup Policy for Windows Server Backup
$BPol = New-WBPolicy

Once we have created our backup policy, we add the drives that we want to backup.  We can tell Windows Server Backup which drives we want to back up by using the drives and folder paths that we retrieved from Exchange using the code above.  We use the Get-WBVolume cmdlet to get the disk or volume information and the Add-WBVolume command to add it to the backup job.

##Define volumes to be backed up based on Exchange filepath information
##Retrieved in function GetStores

ForEach($bvol in $volumes.keys)
$WBVol = Get-WBVolume –volumepath $bvol
Add-WBVolume –policy $BPol –volume $WBVol

The Add-WBVolume doesn’t overwrite previous values, so I can easily add multiple drives to my backup job.

Now that my backup locations have been added, I need to tell WBS that this will be a VSS Full Backup instead of a VSS Copy Backup.  I want to run a full backup because this will commit information in the log files to the database and truncate old logs.  The command to set the backup job to a full backup is:

Set-WBVssBackupOptions -policy $BPol –VssFullBackup

Finally, I need to set my backup target.  This script is designed to back up to a network share.  Since I want to retain multiple backups, it will also create a new folder to store the backup at runtime.  I created a function called AddWBTarget to handle this part of the job.

Function AddWBTarget
##Create New Folder for back in $backuplocation using date format
$folder = get-date -uFormat "%Y-%m-%d-%H-%M"
md "$backupLocation\$folder"
$netFolder = "$backupLocation\$folder"

$netTarget = New-WBBackupTarget -NetworkPath "$netfolder"
Add-WBBackupTarget -policy $BPol -Target $netTarget

The backup location needs to be a UNC path to a network folder, and you set this when you run the script with the –backuplocation parameter.  The function will also create a new folder and then add this location to the backup job using the Add-WBBackupTarget.

The documentation for the Add-WBBackupTarget states that you need to provide user credentials to backup to a network location.  This does not appear to be the case, and WBS appears to use the credentials of the user running the script to access the backup location.

WBS now has all of the information that it needs to perform a backup, so I will pass the temporary backup job to WBS using the Start-WBBackup with the –policy parameter.

You can run the script manually by running EX2k10WBS.ps1 from your Exchange 2010 server.  You will need to declare your backup location by using the –backuplocation parameter.  Since this script will be performing a backup, you will need to run PowerShell with elevated permissions.

You can also set this script to run as a scheduled task.

You can download the entire script here.