File store syncing between two locations

[danielmare]

We have 2 offices connected via a slow internet connection.

They share a 2 TB file store for jobs (e.g. drawings, PDFs, Word documents, etc) shared via SMB hosted on Win Server 2008 R2 and Win Server 2003 servers respectively. The remote office is finding access to it to be very slow.

What I'd like is to put a duplicate in the remote office. Every night, they should sync. If a file is changed in one location and unchanged in the other, the newer version should simply overwrite the older version.

If they're changed in BOTH locations, the old file needs to be copied to a "conflicts" folder, before the more recent file replaces the older one.

I have tried

http://allwaysync.com/,

but this runs out of RAM and crashes when it gets to the analysis of file number 2 million or thereabouts.

Basically I need something like rsync, except I don't think rsync can be set to make backup of files that were changed in both locations. I think there is an option to make a backup of every file that gets overwritten, but I don't want this for files that are changed on one location only, otherwise this backup directory becomes too big too quickly. I just want a "conflicts" backup where they are changed on both locations.

Any suggestions?

 

[itsp1965]

Have you considered using MS DFS?

 

[joepc]

I would look at DFS too but another 3rd party syncing tool that I have great luck with is goodsync. We use this when performing large file server migrations but you can schedule it run as often as you like.

 

[58sniper]

Another vote for DFS. Very good. Only copies the blocks of files that have changed, instead of the whole file again. Clients always pick from the closest DFS source.

Do you have your Tek-Tips.com Swag? I've got mine!

Stop by the new Tek-Tips group at LinkedIn.

 

[kmcferrin]

I vote for DFS too.

I remember in the old days when DFS wasn't very good for this I used Robocopy to accomplish this. I was syncing a large fileshare across a pair of T1 circuits, so I had a Robocopy job that ran every 15 minutes during business hours to sync anything smaller than 10 MB, then it ran another job after hours to sync anything larger than 10 MB. It worked well for the time, but using DFS-R would be far superior these days (and avoid collisions).

________________________________________
CompTIA A+, Network+, Server+, Security+
MCTS:Windows 7
MCSE:Security 2003
MCITP:Server Administrator
MCITP:Enterprise Administrator
MCITP:Virtualization Administrator 2008 R2
Certified Quest vWorkspace Administrator

 

[danielmare]

No, DFS does NOT work. I have wasted the past week trying to get the stupid service to work.

Firstly, it was only replicating one-way... no errors, nothing in the logs to indicate why it was doing that, it just simply wasn't replicating the other way.

I deleted namespaces, replication groups, DfsrPrivate folders, etc, and started from scratch again, except I left the 1TB of already duplicated data on the 2nd server (with the DfsrPrivate folder deleted.)

Recreated namespace containing one folder with the two folders I want to replicate as targets, this automatically brought up the replication dialog and it started replicating...

backlog revelaed 1.1m files, and when I swapped ReceivingMember and SendingMember in the dfsrdiag backlog command, it was just under 1.1m. This 1.1m went down all the way to almost nothing (the opposite direction's count remaining 1.1m) and then when I came in this morning, it was all the way back up to 1.1m again!

I'm now really annoyed that I have wasted so much time on this and it simply does NOT work.

I did find an event 2104:

The DFS Replication service failed to recover from an internal database error on volume C:. Replication has been stopped for all replicated folders on this volume.

Additional Information:
Error: 9203 (The database is corrupt (0))
Volume: D6471C22-AF35-11E1-B122-806E6F6E6963
Database: C:\System Volume Information\DFSR


To get rid of the above error, I started from scratch again. backlog revealed 1.1m files counting down and down until the error came back again and backlog went back up to 1.1m.

I'm now really extremely frustrated and ready to throw DFSR out the window never to be looked at again.