Knowledge Base & Discussion Forum

OBS REPLICATION WITH ROBOCOPY

Discuss technical questions on AhsayRPS

Moderator: Support Team

OBS REPLICATION WITH ROBOCOPY

Postby ag1980 » Sun Feb 05, 2012 2:24 am

Hi We are not having any luck using RPS for replicating our data. Its just too slow - we have about 5TB of data on a single OBS.

we decided to use robocopy for the replication process.

what will be the best syntax for doing incremental backup to our obs users folder with robocopy.

we have created this simple scripts:

robocopy "D:\program files\AhsayOBS and AhsayRPS\user" "\\backup-02\E\user" /MIR /ZB /R:3 /V /FP /M /LOG:sync.log /TEE

does this syntax will do the job ?(will copy over only the changes from obs user folder to rps user folder )
pleas advise ,

thx
ag1980
 
Posts: 3
Joined: Sun Feb 05, 2012 2:02 am

Postby leemason » Mon Feb 06, 2012 1:20 am

I'd be careful using utilities like robocopy (and rsync). If you try to sync the entire OBS server like that it could lead to major issues and the copy may well fall over. OBS creates millions of small files and both robocopy and rsync have some difficulty scanning and sync'ing so many small files.

I did this with rsync and ended up writing a script to scan each user home and sync each individual users home directories separately. That worked much better but requires a bit more scripting.
BackupsAnywhere Ltd (http://www.backupsanywhere.com)
leemason
 
Posts: 670
Joined: Fri Aug 25, 2006 4:35 pm
Location: London, UK

Postby ag1980 » Mon Feb 06, 2012 6:00 am

HI leemason THX for the quick reply,

basically we have two big users folders on the obs.

if i unrestrained you right,the best way to replicate with robocopy\rsync will be to divide the script to multiple scripts that will works at the same time (to create 5\6 jobs for each group of users)

as further i know robocopy and rsync are very demanding software's that taking a lot of CPU performance.
do you have any experience with that?(with rsync)
does rsync\robocopy, can handle with opened\locked files, (vss support)?
ag1980
 
Posts: 3
Joined: Sun Feb 05, 2012 2:02 am

Postby leemason » Tue Feb 07, 2012 2:10 am

You are pretty much correct. If you had, lets say, two user homes you could have a script for each which goes through the user directories under each user home and syncs each individual directory tree.

They are both pretty heavy on resources as you say but doing the above will reduce the load because they are not having to index so many files all at once.

I don't believe that either of them supports VSS and they will probably both have problems with open files but I'm using Centos Linux so that's not a problem as Linux doesn't handle/lock open files like Windows.
BackupsAnywhere Ltd (http://www.backupsanywhere.com)
leemason
 
Posts: 670
Joined: Fri Aug 25, 2006 4:35 pm
Location: London, UK

Postby millhaus » Tue Oct 30, 2012 8:35 pm

Nice and interesting thread!

Pros and Cons of RPS from our experience:

+ Easy to install and to manage
+ Does work automatically, "knows" when a file is no more locked
+ Retention can be defined

- If you stop i.e. the OBS service due to moving userhomes or due to maintenace, replication does stop, too!
--> If you restart the service, it will start again from beginning. No intelligence to restart from where it has been interrupted
- Very slow replication. It is said by Ahsay folks, OBS >= 2,5T can't be replicated very well any more
- If you dismount a LUN from OBS userhome, it will immediately disappear from RPS, too! Even though you have retention!
- Due to snapshot/retention setting, RPS does need more disk space to replicate

....
...
..

I have tested a bit, but still I am wondering about some issues:

1- Would you let robcopy per LUN or per each userhome?
(I.e. LUN1 = robo job1, LUN2 = robo job2, and so on OR LUN1\User1 = robo job1, LUN1\User2 = robo job2, and so on....)

2- Is there a way to reduce the IO load processed by robocopy? I only know the function /IPG:<ms> to increase the gaps between each network packet in order to save some bandwith. But does this function also reduce IO load?

3- If we want to implement a sort of "retention": I thought about using the function /MINAGE:<d> which would only replicate files older than <n> days. But somehow it does only replicate once when started, then it does no more replicate newer files.

4- What about using the function /ZB? "/ZB : Use restartable mode; if access denied use Backup mode" this would restart when the file won't be locked anymore. But I hear this function does produce high load. So better not use with OBS?

Imagine: 1x OBS, approx. 161 userhomes, ~ 9 TByte compressed.

Here's our testing robo script



Code: Select all
@echo off
SET Day=%date:~6,4%.%date:~3,2%.%date:~0,2%

SET _source=e:

SET _target=\\10.10.100.144\e$\rep_test

SET _log=E:\test_%Day%_Replication_Log.txt


SET _what=/MIR /COPYALL /B /MINAGE:2

SET _options=/IPG:5 /NP /NS /NC /NFL /NDL /R:0 /W:0 /RH:0615-1730 /MON:900 /MOT:30 /LOG+:"%_log%"

robocopy %_source% %_target% %_what% %_options%
millhaus
 
Posts: 279
Joined: Tue Oct 04, 2011 9:57 pm


Return to AhsayRPS

Who is online

Users browsing this forum: No registered users and 1 guest

Looking for Rbackup Alternative | Vembu Alternative | Novastor Alternative | Asigra Alternative | BackupAgent Alternative? Try our product.


A wholly owned subsidiary of Ahsay Backup Software Development Company Limited  [HKEx Stock Code: 8290]