Home > Command Line > NetApp File Copy

NetApp File Copy

April 11th, 2009

It always comes up, how can I copy single files, or large areas directly from the NetApp console? Generally the answer comes back, you can’t, use RoboCopy or rsync or another file migration tool. However there are definitely ways of copying files around directly from the filer itself, and often this is the most efficient way of doing it! However, these aren’t the most intuitive or well documented commands.

There may be other methods, and if you have something you have used in the past or you know of, please feel free to share! Not all methods are suitable for all tasks, but each has it’s own individual uses.

ndmpcopy

This is often overlooked as a file / folder copy command, and is often just used to migrate entire volumes around. In fact it can be used to copy individual folders or filers around, and even better can be used to copy data to other filers! Make sure ndmp is enabled first (ndmpd on). The syntax is quite simple…

ndmpcopy /vol/vol_source_name/folder/file /vol/vol_dest_name/file

Just to break this down, we are choosing to copy a filer from “/vol/vol_source_name/folder” and we want to copy it into “/vol/vol_dest_name”. This isn’t too restrictive, we don’t have to keep the same path, and we can even copy things about in the same volume (such as copying things into QTrees if you need). You can copy things from an entire volume, to a single QTree, down to single folders way down in the directory tree. The only real restriction is you cannot use wildcards, and you cannot select multiple files to copy.

If you want to copy files from one filer to another, we simply extend this syntax…

ndmpcopy -sa <user>:<pass> -da <user>:<pass> source_filer:/vol/vol_source_name/folder/file destination_filer:/vol/vol_dest_name/file

Replace <user> and <pass> with the source filer (-sa) login and the destination filer (-da) login. Here we copy a single file from one location on one filer, to another on another!

We can also define the the incremental level of transfer. By default the system will do a level 0 transfer, but you can define to do a single level 1 or 2 incremental transfer. If the data has changed too much, or too much time has passed since the last copy, this may fail or may take longer than a clean level 0.

This can be very useful, and as the filer is doing this at block level, all ACL’s are completely preserved. Take care to enable that the security style is the same on the destination to prevent ACL’s from being converted however.

ONTAP 7.2.6.1 manual page for this can be found – http://now.netapp.com/NOW/knowledge/docs/ontap/rel7261/html/ontap/cmdref/man1/ndmpcopy.1.htm

mv

This is a “priv set advanced” command, and so apparently reserved for “Network Appliance personnel”. “mv” is very straight forward, give it a source and destination, and a single file will get moved. Remember this is a move, so it is not technically a file copy at all.

mv <file2> <file2>

flex clone

This is a real cheat, but a great cheat! You clone an entire volume based on a snapshot, then you split this volume off from the snapshot. This a great way of getting an entire volume copied with minimal disruption. The clone is almost immediately created, and can then be online and used live. The clone split operation happens in the background so you can move things and be live at the new location in very little time at all.

vol clone create new_vol -s volume -b source_vol source_snap

Where “new_vol” is the new volume you want to create, “-s volume” is the space reservation, “-b source_vol” is the parent volume that the clone will be based on and “source_snap” is the snapshot you want to base the clone on.

vol clone split start new_vol

Will then start the split operation on the “new_vol”

vol copy

Rather than a flex clone, if you haven’t got that licensed, you can do a full vol copy. This is effectively the same as a vol clone, but you need to do the entire operation before the volume is online and available. You need to create the destination volume first and then restrict it so that it is ready for the copy. Then you start the copy process.

vol copy start -s snap_name source_vol dest_vol

“-s snap_name” defines the snapshot you want to base the copy on, and “source_vol” and “dest_vol” define the source and destination for the copy. “-S” can also be used to copy across all the snapshots that are also included in the volume. This can be very useful if you need to copy all backups within a volume as well as just the volume data.

lun clone

If you need to copy an entire LUN, and again you haven’t got flex clone licensed, you can do a direct lun clone, and lun clone split. This is only really useful if you need a duplicate of the LUN in the same volume. It will create a clone based on a snapshot that already exists.

lun clone create clone_path -b parent_path parent_snap

“clone_path” being the new LUN you want to create, “parent_path” being the source LUN you want to clone from and “parent_snap” being a snapshot that already exists of the parent LUN. The you need to split the LUN to become independent with.

lun clone split start clone_path

SnapMirror / SnapVault

You can also use SnapMirror or SnapVault to copy data around. SnapMirror can be useful if you need to copy a large amount of data that will change. You can setup a replication schedule, then during a small window of downtime, you can do a final update and bring the new destination online.

dump and restore

This isn’t really a good way of copying files around, but it certainly a method. If you attach a tape device directly to the filer, you could do a dump, then a restore to a new location or filer. This can be the only method if you have a large amount of data to move to a new site, and no bandwidth or no way of having the 2 systems side by side temporarily.

Command Line , , , , ,

  1. Ant
    November 14th, 2011 at 14:30 | #1

    @Chris Kranz
    Hi Chris – I’ve tried the mv command and just get a “rename: no such file or directory” error – I’m guessing my syntax is wrong? “mv /vol/userdata1/staff/dave/file.txt /vol/userdata1/staff/bob/file.txt”.
    Also – have tried ndmpcopy but get a “failed to start dump on source” message.

  2. November 14th, 2011 at 16:56 | #2

    With the mv command, does the destination directory exist?

    I assume it is as you are doing NDMP backups, but is NDMP enabled and turned on on the filer? With the syntax, did you put the full path of the NetApp and the volumes? I’ve done “ndmpcopy” to copy between volumes, but I’m not sure if I’ve ever done it to copy within the same volume. Worth checking the syntax, it should still work however.

  3. Ant
    November 15th, 2011 at 08:38 | #3

    @Chris Kranz
    Yes – the destination directory definitely exists, and yep, I’ve checked ndmp is on and happy. I haven’t been putting the full path of the netapp and the volumes. Should it be in the format “myfiler:\vol\volumename\directory\file”?

  4. November 16th, 2011 at 11:45 | #4

    Should that not be “/” instead of “\” ?

  5. Ant
    November 16th, 2011 at 12:34 | #5

    @Chris Kranz
    You’re quite right.

  6. Robert Edgel
    December 6th, 2011 at 20:37 | #6

    Also check out the “clone start” command. This command creates a “pre-deduplicated” copy of any file within the active filesystem using the underlying SIS technology of Ontap (7.3.x and above I believe – flexclone license required). I deploy all my VMWare and Hyper-V VMs this way and get an immediate copy from a template that can be brought online instantly and consumes no space initially (until the blocks diverge over time). Of course, the source and destination file must live in the same volume for this to be useful (otherwise use vol clone). At NetApp Insight, there were several sessions that focused on SIS based cloning and how it will become an ever more standard part of how NetApp performs cloning and even recovery tasks in Ontap 8.1. Today, I cannot create a clone from a file in a snapshot (only the active file-system), but that limitation is overcome in the Ontap 8.1 timeframe.

  7. Harry Schuetz
    June 19th, 2012 at 08:44 | #7

    Hi Chris,
    your post helped me to restore LUNs from tape backup to the filer.
    I’m restored the files to a new location on the filer, where the library is connected and than I copied the LUN to the destination filer.
    Now I have the problem, that I can’t delete the LUNs in the restore folder. I’m get the error “permission denied”.
    How can I delete it?
    Best regards,
    Harry

  8. June 20th, 2012 at 18:59 | #8

    Have you checked the permissions on the volume? If this is all correct (NTFS is you are access from Windows and you are a domain admin, UNIX if you are accessing from *NIX and root), then you could try the “rm” command under “priv set advanced”, or simply delete the volume with the LUN in it (assuming it’s not a shared volume?)

  9. Pavan
    October 17th, 2012 at 19:22 | #9

    Hi Chris,

    I need to migrate the Volume from one Aggregate to differetn one
    That Volume has got Single Qtree which has been shared to Unix systems

    Is it Preferable to migrate it using Vol Copy / ( will that be consitent )

    or shall I proceed with NDMP Copy

    Which one will be seemless ,

    Thanks
    Pavan

  10. abhishek mishra
    November 15th, 2012 at 13:21 | #10

    i have to make the backup of data available on NAS on a diskarray which is connected to a solaris server.

    Is there any faster way other than simple copy , to copy data from netapp to the diskarray connected to the server.

  11. December 12th, 2012 at 10:03 | #11

    rsync? dump? There’s no tools from NetApp that will let you do this, so you are going to be doing host based copies I’m afraid.

  12. December 12th, 2012 at 10:06 | #12

    Neither will be seamless (unfortunately), depending what version of ONTAP you have you could use Data-Motion (a combination of SnapMirror and clever remapping). My preferred NetApp-NetApp copy is usually SnapMirror, but failing that usually NDMP copy.

Comment pages
  1. October 11th, 2010 at 10:29 | #1
  2. February 2nd, 2011 at 22:01 | #2



This site is not affiliated or sponsored in anyway by NetApp or any other company mentioned within.

© 2009-2014 Chris Kranz All Rights Reserved
This site is not affiliated or sponsored in anyway by NetApp or any other company mentioned within.

%d bloggers like this: