Sustained throughput in a Legato NetWorker environment

[fraxx]

I'm toying with a scenario where I want to backup several terabytes
daily in a NetWorker enviornment.

Some facts,
Clients: several hundred of many different OSes
Data volume: about 20 TB needs to backed up during a lot less than 10
hours, I'll probably aim for 5 hours. That's 4 TB/hour or approx 1120
MB/s.


I plan to use one or more virtual tape libraries as a storage device,
each one can give me 300 MB/s sustained write speed.


I plan to use several dedicated storage nodes. The storage nodes are
backing up the clients over the LAN. I will not use a dedicated LAN
segment, I don't see any point. With clever placements of storage
nodes I can pull the clients over the backplanes.


The storage nodes will be centrally managed from a NetWorker master
server, but the master server will be busy handling index and I don't
want to use the master to move data as well.


Question is, how many storage nodes do I need to have a combined
throughput of 1120 MB/s. I can probably trunk a few GbitE connections,
but do I trunk 2 and hope for ~150 MB/s or can I go with even 4 GbitE
and hope for ~300MB/s. In any case I want to use x86 hardware and
scale out instead of buying some huge box and scale up. OS on these
boxes is a pretty open subject, but something tells med Linux would a
be good choice. These boxes are truly dedicated for the task, they
will not be running any other applications.
Could I hope for a 4x Gig trunk on a 2 CPU machine, some AMD dual core
variety are dirt cheap these days. On the backend I'll use 4Gbit Fibre
Channel to stream the data to the VTL.

 

[creovisual]

You will not get 150 MB/sec from 2 Gbit NICs. From my experience, you will not see more than about 50MB/sec from a single NIC (sustained) and if you trunk 2, you may see close to 100MB/sec. This number will fall short of what an average 2 way (4 core or better) comodity 2U server (Dell 2950, HP DL380 G4/G5) can mange however. I have seen these servers sustain 200+ MB/sec. I would recommend trunking 4 (or more, Gbit is cheap relatively speaking) Gbit NICs to provide suitable throughput (unless the server is underpowered).

At these numbers, you would need something on the order of 6 servers (give or take) to get you in the ballpark of 1100 MB/sec, but in practice I would anticipate the real number to be closer to 8. This would account for less than ideal balancing of the load between the storage nodes, and even at that you will not likely see a sustained throughput at 1100 MB/sec throughout the entire backup window, unless you have very predictable data and clients that can maintain transfer rates...

Cheers.

 

[a060463xyz]

first of all a dedicated storage node is only capable of backing up it sown data, for your szenario you need non dedicated Storage nodes.
I think your szenario has some critical points :
with several hundred clients it might be better to use more than one networker datazone because the sum of all cfi should not exceed 200 GB. So if your browse policy is relativly long you might need more than one networker Server.
VTLs are fine but in your szenario using snapshots with the networker powersnap module is preferrable. Therefore you need a storage consolidation first.
EG. if you use netapp filers for your storage you can get high backup transferrates using the Networker NDMP-Option.
Taking in concern the huge Number of Clients and Terabytes i advice you to look for an expirienced Storage Consultant to work on the design of your backup infrastructure.

 

[fraxx]

Ok, I used the term dedicated in meaning a server that is a storage node only, and is not running any other application(s). I find the NetWorker terminology confusing, but I do mean from a licensing viewpoint a full Storage Node.

Retention is for the most part only a month, but indeed it's a question that's come to mind. However, I want to keep the number of zones to a minimum, and one zone is certainly the bare minimum.

The storage platforms... name it, we've got it.

And the experienced storage consultant, well... guess that would be me.