CIFS data migrations
Almost seamless! Sort of…
As with most of my thoughts, it started with an innocent customer query. EMC have some very cool inbuilt tools for doing seamless CIFS data migration, but NetApp don’t. It’s something that often causes a fair amount of problems and some careful planning with NetApp as we don’t have this. But I was thinking today, we kinda do, I just don’t think we leverage the tools available properly.
Enter widelinks. Here is an excerpt from a NetApp KB article on the topic (KB 3011420)…
A symbolic link is a special file created by NFS clients that points to another file or directory. Widelink entries are a way to redirect absolute symbolic links on the filer. They allow the symbolic link destination to be a share on the same filer or on another filer. The following examples illustrate how to create a symlink from volume to qtree on the same filer, and from volume to volume on different filers.
What does this mean and why will my life be easier after reading the rest of this article?
So if I have a nice shiny new NetApp filer (or an old one I haven’t got round to migrating my CIFS data onto yet), and I have my old CIFS file server that is rapidly approaching failure or out of support. I can create my new file and share structure on my NetApp, and then use widelinks to redirect the user to the CIFS file server while I worry about all the data copy out of hours without having the ball-ache of copying all my data all at once.
First of all let me highly recommend that if you aren’t using DFS, start using it. You’re going to have to repoint your users to a new share name anyway, so you might as well do it properly. Setup a basic DFS root and repoint all your users to this. Why? Because next time you come round to reconfiguring or upgrading your CIFS share infrastructure you won’t have to touch your users, just repoint DFS one night.
So widelinks expands on symlinks. Don’t get too excited because at the moment “mklink” from Windows 7/2008 isn’t the same thing, and it won’t work on NetApp shares. Symbolic links in this article are a *NIX thing, but the concept is similar to Windows symbolic links. If you don’t know symbolic links, think of them as shortcuts, they simply redirect you to another location without actually redirecting you. What widelinks will do is translate this symbolic link and bridge the gap to the remote location.
Okay, enough theory, onto the practice…
My setup is quite simple. I have a NetApp simulator running as a cluster, I have a NAS box that has some home media, I have a MBA to do some of the UNIX stuff I need (sorry, I tried to get round this, but a Linux/UNIX box is needed for now). Filer1 is my new system that I want all my users to use. Filer2 (with a folder called test2) and my NAS box (with movies) are legacy systems I’m looking to replace (bit of role-play).
First of all my CIFS share on the new system is created. We need to make sure this has “widelinks” enabled, so the command line is as follows…
cifs shares -add /vol/test1 test1 -widelink
If you already have your shares setup, no problem…
cifs shares -change <share_name> -widelink
If you query the cifs shares, you should get output similar to below…
node1*> cifs shares
Name Mount Point Description
- – - – - – - – - – - – - – -
. . . widelinks supported
everyone / Full Control
I can see that widelinks is enabled for my share. I need to export this as NFS to my MBA so I can create the symbolic link. Before you ask, I tried both Adam Fox‘s “ntap_symlink” and Oliver Krause’s “ln”, but these didn’t do what I wanted (ln just crashes on Win7). I’m not sure if you can mount the target for the symbolic links as anything other than NFS, but it can certainly map to file systems that aren’t just NFS, I have done this also with an external CIFS share.
exportfs -io root=<unix_hostname> /vol/test1
From my MBA I can then mount this up and create some symbolic links. (I already have my NAS mounted)
mount filer1:/vol/test1 /mnt/test1
mount filer2:/vol/test2 /mnt/test2
ln -s /mnt/test2/test2 test2
ln -s /Volumes/movies movies
If you “ls –lah” the folder you’ll see the symlinks created and it should show the mappings. Job done for my MBA, back to Windows and the NetApp (phew you sigh).
We need to create a translations file for the symlinks. Basically this reads the symbolic link that we created on our UNIX host and converts this to a DFS style link to redirect the data path. So make sure you noted the paths you used on your UNIX host for the mappings! The “*” in the paths are useful as you can include many different symbolic links here and they will all be matched and followed.
widelink /mnt/test2/* \\filer2\test2\*
widelink /Volumes/movies/* \\NAS\movies\*
This file is re-read every 30 seconds, so patience, go get a glass of water. Go back to the test1 CIFS share and you should see some magic! (test1 already exists as new data).
More magic is shown when you right click one of these folders and go to the DFS tab. (you’ll also see a DFS tab on folders that aren’t widelinks, but you’ll notice the referral list is just the normal share).
Additionally you’ll notice that if you go to the “test2″ folder you’ll see all its contents without you being actually redirected to this other host.
Now we have the challenge of data migration. The advantage (and major problem) of widelinks is that they appear and feel to a Windows user as data on the share they are looking at. So users can’t edit the symbolic link, or over-write the data (which is probably a good thing). If they copy over the folder, it’ll attempt to update the target (in our case, the legacy systems). But as administrator you have the same constraints. So what do I do???
Unfortunately this is where the cool stuff things break apart a little. What I need to do is remove the symbolic link, remove the entry in “/etc/symlink.translations” and then copy my data across to fill the space. When doing this make sure to make the old data unavailable (change the share permissions is safest and simplest) as the symbolic links can be a little sticky and you don’t want users writing to 2 locations. I had some odd results with this on my Windows 7 desktop as it seemed to cache the widelink, but mapping the drive from a different machine worked fine. I guess you’ll want all clients offline when you do data copies.
Now wouldn’t it be cool if widelinks could be some way integrated into the OSSV mechanism? In case you don’t know, OSSV can do file copies from a Windows host onto the NetApp into qtrees. This would make a very nice migration tool!
Please NetApp consider the following improvements to widelinks:
- Allow me to create symbolic links from Windows (mklink) – this is a must have! All users out there, log a support call regarding this, the more people request it, the quicker they’ll do it!
- Give me a mechanism to integrate this with OSSV (I know I’m asking a lot)
- Once you’ve integrated OSSV into this, give me a mechanism to transparently remove the widelinks once my data has copied across. This surely isn’t asking the impossible as ndmp copies dump the inodes and ACLs last and that’s what needs to be achieved here.
- A GUI to manage my widelinks and symbolic links. I toyed with the idea of creating a PowerShell to manage this, but the issues of creating symbolic links aren’t easily achieved on Windows.
Special thanks to Adam Fox as a few of his topics on the communities pointed me in the right direction. Useful references included below: