NET_BUFFER_SZ values for Gigabit ? 1

[FredTheCat]

I have a NetBackup Data Center environment with a Win2K master, a Tru64 Unix media server, and a handful of clients on Open VMS, HP UX, Win2K.
We have already moved the VMS clients, the Win2K Veritas Master, and the Tru64 Unix media server from 100 BT ethernet to gigabit connections. The adapters and switch ports have been set to allow Jumbo packets (mtu = 9000).
Now, I understand that I have to tweak the value of NET_BUFFER_SZ.
Here are my concerns:
1) Some clients will not upgrade to Gigabit due to hardware restrictions, and I don't want to ruin the backup performance on them while improving it elsewhere.
2) I have not played with the SIZE_DATA_BUFFERS or NUMBER_DATA_BUFFERS, and am wondering if they will have to be bumped up as well, to get the best throughput.

Has anyone had experience adjusting these parameters, and either some guidelines to follow and/or some numbers for these values from their own environment to use as a comparison?

 

[PGPhantom]

Wow - I am surprised that you are not having a lot og "UNKNOWN" network issues. The default and recommended MTU setting from Microsoft for Ethernet networks is 1500.

Verify your settings ...
To calculate “ping server -f -l <Packet Size-28k for IP and ICMP headers). When size is exceeded the packet will get broken up.

Check my FAQ at

http://www.tek-tips.com/faqs.cfm?spid=776&sfid=3124

- This should give you the settings that I have used to get maximum throughput in a Windows environment.

Also - Make sure that you have flow ontrol set to Rx/Tx, Checksum Offloading set to RX/Tx on switch and NIC.

Let me know if this helps

 

[FredTheCat]

Thanks, I know about the default /recommended MTU of 1500 for traditional Ethernet... but Gigabit is recommended to run at 9000 MTU, if the servers and network components (switches) are all set to support jumbo packets.
We have done the appropriate config to support jumbo frames.

Using your ping test above, I quite easily got above 7500 with no lost packets.
...But perhaps I am not getting the 9000 MTU per spec, since I get dropped packets if I go much higher.
I am reading through your FAQ and it looks like it might steer me in the right direction. I'll let you know.

 

[FredTheCat]

PGPPhantom's FAQ was a lot of help in understanding how the various settings (net_buffer_sz, size_data_buffers, number_data_buffers) fit together and to give some reasonable baselines.
However, there seems to be a knowledge gap out there regarding using Gigabit, especially regarding jumbo frames. Even the vendors' tech support staff have to dig for answers rather than just pull it out of their knowledge base.

My main obstacle now is, my network connections can send faster than an individual DLT drive can write... so I have to allow data multiple streams so I am writing to more than 1 tape drive at a time.
There also seems to be an MTU limit of 7532 on VMS boxes, and 9000 on Unix and Win2K. ... if anyone has run into this limit on VMS and found a way around it, I would appreciate hearing about it.

More tweaking and tuning...such is life !

 

[PGPhantom]

The main reason that I did not play around with the MTU's and please keep this in mind, Your entire network must be set at the same rate. All switches, NIC's etc - If there is any discrepency, you will have dropped packets etc and performance will be negatively influenced rather than positively.

With 200 plus servers in multiple locations, it just is not feasible for usto change the default MTU's - In a smaller environment - No problem.

 

[FredTheCat]

I don't wish to be argumentative, but I think someone has steered you wrong regarding a mixed environment. Perhaps it was true with earlier switches and drivers.
To date we have 2 VMS servers, 1 Tru64 Unix server, and 2 Win2K servers all on Gigabit connections running with jumbo frames enabled on the adapters and on the idividual ports on the Cisco switch, networking quite cleanly with a variety of desktop clients, plus lower-end servers on VMS, HP UX, Win2K, etc. ... and if anything, network calls such as FTP, BCP, telnet, NetBackup (of course), etc have improved rather than degraded.
Just about every other server is running 100 BT full duplex, and the desktop clients are a mix of 100 and 10 BT, some local and some across the country.

What can be a BIG gotcha, is if you do not (or cannot) configure the individual ports on the switch to enable/disable jumbo frames in accordance with the type and settings of the adapter connected to it. When first installing, we had enabled jumbo frames on all ports to which the gigabit cards were to be attached, and had nasty issues with timeouts and dropped packets until the adapters and switch ports matched configuration.
Getting back to where this thread started, we are quite satisfied with the performance now within NetBackup and without, and the FAQ referred to was quite on target. Backups that used to take 90 minutes over LAN now complete in about 20 minutes.
However, your concerns regarding jumbo frames, though doubtless with the best of intentions, do not seem to be accurate from our own direct observations. All of the other (100BT of lower) elements in our WAN are communicating with the upgraded servers as well or better than they did before.

Thanks again for your earlier help.

 

[PGPhantom]

Thank you - You are correct and I should of elaborated a bit more ...

MTU's can be different and from my understanding (I welcome corrections if I am wrong) the communications between two servers will be at the lowest MTU of the 2. If you have server1 at 1500, server2 at 2000, server3 at 3000 and server4 at 3000. If 2 and 3 communicate, you will get 2000, if 1 and 2 communicate you will get 1500, if 3 and 4 communicate you will get 3000.

This does assume that switches are set and support the same Jumbo frames. I tried to keep it simple by saying everything must be the same to avoid confusion, but that in itself is confusing and misleading.

Thanks for the correction.