Home > General > Deduplicating LUNs

Deduplicating LUNs

January 18th, 2009

This is primarily focused at VMware usage, but in theory now that SnapDrive 5/6 support thin provisioning of LUNs, it could be applicable elsewhere. I’m not sure on the support / affects with any SMAI product. These are untested and I would recommend to try build this in to the demo / test lab somewhere to check how it works and any potential impact / issues.

 

DeDuplication on LUNs that have no thin provisioning will save you space in only one place, Snapshot usage (going forward that is, already existing snapshots won’t get deduplication as they are read-only).

 

 

TR 3505 (http://media.netapp.com/documents/tr-3505.pdf) talks about FAS DeDuplication. Chapter 4.10 (page 29) talks about the setup of LUNs. Different settings free up the storage in different locations. The only way to free up the storage within the Aggregate however is to thin provision or shrink the volume. The LUN can also be non-space guaranteed to reduce the overhead there. I wouldn’t necessarily recommend changing the Fractional Reservation (although entirely possible if you have smaller thick provisioned volumes and thin provisioned LUNs) at the same time as everything else, but keep an eye on it and monitor the rates of change to see if you can.

 

Previously I have always been concerned with using volume auto-grow and snapshot auto-delete as they haven’t been an exact science and have often failed with LUNs as we weren’t sure what would trigger a volume usage limit. However the TR is now based on this, and the NetApp 50% VMware space saving guarantee relies on this (pretty much). If you don’t set these and you run out of space, (either through excessive change in snapshots, or excessive data growth), then your LUNs get switched offline to preserve their data (see other posts on Fractional Reservation).

 

To prevent this we need to enable both volume auto-grow and snapshot auto-delete. NetApp best practice try-first is volume auto-grow and I would agree with this. You don’t want a large rate of change, and then to come back the next day and find all your backups gone wrong. What if you got a virus or some form of data corruption that caused all the data to change? This can also mess with SMAI backups and any replication you have set in place.

 

This is going to be tricky to size. We have a NetApp guarantee of 50% space savings within VMware, so in theory we can half the storage requirements. Remember dedupe is still post-process, so if this is a one step migrate, then start dedupe, we will need 100% storage available for the migrate. The LUN needs to be formatted to the correct size so that VMware sees all the available storage, LUNs still can’t be dynamically sized (and we still don’t talk about extents). If this is a new environment, or a gradual migrate, we can be a lot more aggressive with the sizing.

 

 

I’d like to add an extra comment also, the DeDuplication process is on the storage array, so if you are assigning block-based storage (iSCSI / FCP LUNs), then these are given to another operating system to format. There will be no space savings seen from the host. So if you can deduplicate 500G worth of Virtual Machines into 5G, you’ll still need to see 500G within VMware, and you’ll still see this full!

General , , , , , , ,

  1. No comments yet.
  1. No trackbacks yet.



This site is not affiliated or sponsored in anyway by NetApp or any other company mentioned within.
%d bloggers like this: