What Does the Software-Defined Data Center Suite Mean For Managed Services Providers

If one thing has been made clear by the general sessions at this year’s VMWorld, it’s that the cloud is now here to stay, and VMWare and other vendors are providing tools to manage the cloud, where ever it might reside, and the machines that run on it.

The second general session of this year’s VMWorld focused on two tools in the vCloud Suite:  vCloud Automation Center, which handles infrastructure and application provisioning to turn IT into a service, and vCloud Operations Management, which handles monitoring and remediation of problems.  These tools, as well as some other tools in the vCloud Suite, tie in closely with both vSphere and other cloud providers like Amazon Web Services and Microsoft Azure to provide automated provisioning and management of public, private, and hybrid clouds.

As the presenters were demonstrating these products and showing how they worked together to deploy and maintain applications, I started to wonder what this meant for managed services providers who’s product was managing IT infrastructures.  These companies tend to focus on small-to-medium sized entities that don’t want to take on the additional expenses of staff, IT monitoring, or 24–hour operations.  Can this software replace these providers?

If managed services providers can’t find ways to bring additional value to their customers,they will be quickly replaced.  If software has gotten to the point where it can not only detect an issue but attempt to remediate it as well based on policies that the administrators set or perform a root cause analysis immediately to pinpoint the issues so administrators can act, then there is a significant cost savings that can be captured on the customer’s side.  Even if taking advantage of the advanced remediation provided by these software packages requires a little work to get right, the ongoing cost savings that would be generated make this sort of investment very attractive.

At $work, we currently use a managed service provider.  They provide monitoring and patching for the most critical servers in our environment, which comes out to about one third of our environment.  The rest are managed using a variety of tools such as the monitoring in vCenter and scripts.  Like many environments, the monitoring coverage is not ideal.

But when I look at the cost of expanding managed services to cover the rest of my environment, or even continuing it, and compare it to using a software solution, there’s no contest.  I can get a greater level of coverage, some level of automated remediation and intelligent baselining, and a short payback period.

Now, I realize that this won’t be implemented overnight.  These systems can be just as complex as the infrastructures they are monitoring, and they take time to learn the network and develop baselines.  But the payoff, if done right, is software that goes beyond monitoring systems to managing them for you.

Infrastructures are going to get more complex now that software-defined storage and networking are a reality and vSphere is getting application-aware features.  If managed services providers want to remain relevant, they need to bring more value to their customers, update their tools and their offerings to better support the cloud, and work more closely with their customers to understand their environments and their needs.

If they don’t, then their customers will be throwing good money after bad.

Utilizing Offsite Backups to Seed Backup Copy Jobs in @Veeam #V7

In the last couple posts about Veeam, I mentioned that $Work has been doing backups directly to our offsite storage.  Due to limits on bandwidth, any errors, changes, or server additions can have a drastic impact on our ability to complete backups in a timely manner.  And if you’ve ever tried to do a full backup of a 1.5TB file server over a 10Mbit connection because the VMID changed, you’ll know exactly what kind of pain I’ve felt in the past.

While a backup copy job will eliminate some of this pain, it still needs to be seeded at the remote site. 

I was a little disappointed to learn that an existing backup chain cannot be the target of a backup copy job.  The backup copy job needs to be pointed to a clean full backup that doesn’t include any forward or reverse incremental backups.  I’m not sure what the reason for this is, but it was confirmed in a Veeam in this thread on their support forums.

But there is a workaround to this, and it’s fairly easy.  The process is actually pretty simple.

  1. Create a backup copy job.  Use the backup job that currently saves to the remote site or the virtual machine as the source and use a backup repository at the remote site as the destination.
  2. Let the Backup Copy job run and create a new full backup of the file.
  3. Once the Backup Copy job has successfully completed (see notes below), create a new backup job for those servers and store that backup data at your local/primary site.  You cannot change the target of your existing backup job –  Veeam will require you to copy your existing backup chain to that location.  Its easier, and faster, to just create a new backup job.
  4. Edit your Backup Copy job to remove the job that backs up directly to the remote site and add the job that backs up to your primary site.
  5. The next time your VMs back up inside the copy window, it will sync the changes from the latest restore point to your remote site.

There are a couple of caveats with this.  You can’t set up a new backup chain at your primary site until you’ve created your backup copy job and created the new full backup file.  If there are more recent restore points available than the ones at your remote storage site, it will eschew the ones at the remote site in favor of the ones at your primary site.  This may mean copying a large amount of data over your WAN.

Second, you need to check to make sure that all of your data has copied over.  A copy job may end succcessfully if the interval expires and it copied some data.  If it hasn’t finished copying all of your data, it will restart and pick up where it left off.  This might give you a false sense of data security by making you think that your offsite backups have fully seeded when you still have to copy large amounts of data over your WAN.  In this case, it would be helpful to have a warning on any job notifications to inform the administrators that the seeding hasn’t completed and that there isn’t a restore point in the remote repository yet.


@Veeam #V7’s Killer Feature

Of all the features that have been added to the latest version of Veeam, there is one that really stands out as the killer feature.  This feature is available in all of the licensed versions of Veeam, and there are no restrictions on the base functionality.  This feature wasn’t widely heralded from what I can tell.

That feature, in my opinion, is the Backup Copy Job.  As I mentioned in my last post, I wanted to dedicate a little more time to this feature.

But I have a confession to make.  I want to make it clear that I don’t know what features are in Commvault, NetVault, Symantec, Avamar or other backup solutions.  Similar features probably do exist.  I do know that other software vendors have had GFS rotation for a long time, but I don’t know enough to say how it ties in with their virtualization backup or their offsite capabilites.  I also just want to focus on Veeam’s implementation.

So why am I making a big deal out of this if I think other vendors may have this capability?  Because this is what a lot of customers have been asking for for a long time.

In previous versions of Veeam, you couldn’t do any sort of backup rotation.  Well, that’s not entirely true.  It would be more accurate to say that there wasn’t any built-in functionality for doing GSF rotation, and their support forums have a number of hacks that add this capability by using PowerShell or recommending multiple backup jobs on varying schedules to handle this.

By setting up GFS rotation and building it into a new method for utilizing offsite storage, Veeam has built a powerful tool for backing up virtual environments and ensuring that your data is safely protected offsite without having to break the bank on expensive backup storage.

First Thoughts on @Veeam #V7

Veeam released the latest version of their backup software a week ago on August 15th.  I’ve been looking forward to this release as they’ve included some features that many customers have wanted for some time such as:

  • Grandfather-Father-Son backup rotation as part of a Backup Copy Job to secondary storage
  • Export Backups to Tape
  • vSphere Web Client Plugin
  • Built-In WAN Acceleration

The full list of enhancements and features can be found here.

$Work uses Veeam as the primary backup solution, so I set up a test environment to try out some of these new features before upgrading.  $Work is only licensed for the Standard Edition, and while the evaluation license is for the Enterprise Plus feature set, I will only be testing what I can use in my production environment.  So unfortunately, I won’t be trying out the WAN Acceleration feature or U-AIR.

First Thoughts

Installation of V7 and setting up jobs was a breeze.  There were a few small changes to the process compared to previous versions, like having to set up credentials to access VCenter and Windows servers in a credential vault, but those changes were relatively minor and saved time later.  In previous versions, I would have to go into my password vault each time I wanted to create a backup job that included windows servers.  This takes care of that.

Not much has changed with setting up new backup jobs.  They have added a screen for setting up a secondary storage site and backup rotation, which makes it easy to add backup jobs to a backup copy job if you already have one set up.  One of the best changes on various jobs screens, in my opinion, is that the backup job statistics screen is now accessible on the main screen just by selecting a backup job.  It is no longer buried in a context meu.

Previous versions of Veeam backed up servers sequentially if there was more than one server per backup job.  That’s changed in this edition.  Veeam will now backup multiple servers per job in parallel.  This will cut down backup times significantly.  This option isn’t enabled if you are upgrading from a previous version, but it can easily be enabled by going into the options menu.

I really like the Backup Copy job option.  There is a lot to this feature, and I want to dedicate more time to it in a separate post.

The timing of this release is very good.  We are a Veeam customer at $work, and we’ve just started to reevaluate our disaster recovery plan and capabilities.  Some of these features, especially the exporting backups to tape and GFS rotation, are capabilities that we wanted to get.  We currently back up directly to an offsite repository, so the backup copy job feature may be one of the best additions to this product.

My VMworld Schedule

I’ll be attending my first VMworld at the end of the month.  My schedule focuses mainly on three areas:  Automation (PowerCLI and Puppet), VMware View, and networking.


VSVC4944  —  PowerCLI Best Practices – A Deep Dive 11:00 AM

NET1000-GD  —  vSphere Distributed Switch with Vyenkatesh Deshpande 12:30 PM

VAPP5613  —  Successfully Virtualize Microsoft Exchange Server 2:00 PM

VCM7369-S  —  Uncovering the Hidden Truth in Log Data With vCenter Log Insight 3:30 PM


SEC5755  —  VMware NSX with Next-Generation Security by Palo Alto Networks 11:00 AM

EUC4764  —  What’s New and Next for VMware Horizon View 1:00 PM

EUC5434  —  Enterprise Architecture Design for VMware Horizon View 5.2 2:30 PM

BCO5362  —  Veeam Backup & Replication v7 Deep Dive 4:00 PM

VCM5271  —  VMware and Puppet: How to Plan, Deploy & Manage Modern Applications 5:30 PM


VSVC5931  —  PowerCLI What’s New? Administrating with the CLI Was Never Easier 8:00 AM

EUC5249  —  PCoIP: Sizing For Success 10:00 AM

VSVC5511  —  Deploying vSphere with OpenStack: What It Means to Your Cloud Environment 11:00 AM

VAPP5932  —  Virtualizing Highly Available SQL Servers 12:30 PM

EUC4629  —  ThinApp 101 and what’s new in ThinApp next version 2:30 PM

EUC1006-GD  —  View with Andre Leibovici 4:00 PM


I don’t have any Thursday sessions on my schedule.  I have two good reasons for this.  The first is that the few sessions that I wanted to attend were already full.  The other reason is that I plan on sitting for the VCP on Thursday morning.  I plan to spend whatever time I have left after that talking to vendors or visiting a friend in San Diego.