When you are using VMware esx with the enterprise features you have to store your Virtual Machines on shared storage like Netapp provides. As everybody knows your storage usage will explode immediately. It isjust too easy to create multiple virtual machines that will eat up your storage space.
At the VMUG 2008 event I took a look on the presentation from Eric Sloof where he was showing a power shell script which can batch create multiple vm’s in a minute with one mouse click. You can imagine what this script will do for your storage environment.
Netapp added a big value to VMware storage utilization by providing De-duplication. Especially in a VMware environment where VM primary partitions are the same for about 80%. De-duplication scans these volumes where the vm’s resides and removes all the double blocks and create pointers to them.
The fun thing about De-duplication is the answer on what does it cost on Netapp? It is free! At least for the latest FAS systems. It is not available to R100, R150, FAS250, FAS270, and products that are within the range of the 800 or 900 series.
To give you a quick start on the usage of de-duplication I wrote ten tips on using it in your VMware environment:
1.) De-duplication is a scheduled process. Default it will run every night. So when you do storage Vmotion from a non de-duplicated volume to a de-duplicated volume it is not de-duplicated immediately. Be sure you have calculated this before you do a storage vmotion
2.) Be sure to set “fractional reserve” in a de-duplication volume not at 100%. The reason is that when you de-duplicate the volume, all free space will be added to the fractional reserve pool. So you never get any in the volume where your data resist. When you set fractional reserve at 20%, only 20% of the free blocks are added to “fractional reserve pool and the other 80% to your volume free space!
3.) Use NFS volumes for your VM’s. NFS is far the most flexible storage provisionary for VMware. You can thin provision VM’s and resize NFS volumes on the fly when you want to use De-duplication space savings for another volume.
4.) Place your VM swapfiles on a different Volume. It is best practice for VMware esx environments to place your swap files on a dedicated swapfile volume. Swap files do not have much identical blocks which cab de-duplicated.
5.) Be sure that everyone who has access at the volumes to understand what De-duplication does and what they need to know what happens with data that migrates or is being copied to other volumes/clients. This could prevent volumes to go offline or other data corruption, because it is un-de-duplicated when copied to unsupported storage.
6.) Schedule the De-duplication process at time frames where there is no data access like production or backup windows.
7.) Be sure to not create a Netapp snapshot while the de-duplication process is running, because the metadata can be locked inside the snapshot. (this data is within 1-3% overhead) Instead create a snapshot before de-duplicating. Afterwards Snapshots are not de-duplicated!
8.) Use the best practice for volume provisioning to ESX environments. Provide a 500 GB volume as the maximum. This will always let Netapp De-duplicate your storage on any FAS system. This is also a best practice for ESX performance.
9.) Always place the same OS vm’s in the same volumes. Create volumes per OS. This will free up more space in your volumes.
10.) To get the most out of your De-duplicated space savings, split up OS disks and data disks on different volumes. More duplicated blocks in a volume give you more de-duplicated space!
Commands to start with de-duplication:
First to enable De-duplication on a Netapp volume:
sis on “Volume name”
Next to start a De-duplication process for the first time. (after this a default schedule will de-duplicate your volume every night.
sis start -s “Volume name”
To start the process manually:
sis start “Volume name”
To display the status of the process:
sis status [-l] “Volume name” (in advanced mode)
To see your space savings:
df –s
To stop your De-duplication process. (It will stop your schedule as well).
sis stop “Volume name”
To turn it off:
sis off “Volume name”
To un-De-duplicate your Volume:
sis undo “Volume name”
Ten Dam 
Very nicely said.I noticed your article from Bing and enjoyed reading it. How long have you kept the blog?Just the other day I recently started a blog on my own and its been a enjoyable process. I’ve met some new people since then but it can be a real chore sometimes! Oh well, thanks a million for your blog post!
Pingback: VMware/NetApp/Deduping « My Work Blog
I’m curious about your tip #2, how does setting the fractional reserve affect the NFS volume? fractional reserve is only for LUNs so it would come into play if you were using fiber channel and VMFS datastores not NFS datastores in VMWare correct?
Hi Michael,
You are right about fractionel reserve and NFS. The article was written when NFS was expensive en ISCSI was used as datastores more often. As you know ISCSI is a LUN so Fractional reserve had to be disabled. At that time(2008) fractional reserve was set on by default. Now this setting is not recommended any more and is not set by default at luns.
Tomas