Virtual Ethernet Speed - how fast is yours?

[jprabaker]

I'm doing some copies over virtual ethernet and only get around 4Gb per minute, coping a 350G oracle database. This doesn't seem that fast. Anyone got it working faster?


cheers,
John

 

[p5wizard]

Did you enable jumbo frames?
I don't know speed by heart, will look into it tomorrow.


HTH,

p5wizard

 

[jprabaker]

Jumbo frames isn't an option on my virtual ethernet adapters. Never used Jumbo Frames but I thought it was set against the ethernet adapter.

We are at ioslevel 1.3.0.1 FixPack 8.1

 

[p5wizard]

You just set the MTU to a high value, if it is allowed on your switches if the Virtual ethernet is bridged to real ethernet via the VIO server SharedEthernetAdapter.
For a plain virtual LAN between LPARs of the same p5, you can just set it to 9000 .


HTH,

p5wizard

 

[jprabaker]

Interesting, thanks. Do you need to set anything else such as sb_max, or tcp_recvspace/tcp_sendspace on the interface?

Would you use a 9000 MTU on the VIO servers SEA? My SEA's are linked through etherchannel adapters...

I'm also looking at setting up Jumbo Frames on some standard etherchannel link aggregates. Are there any additional settings in this setup?

Thanks,
John

 

[jprabaker]

The Maximum MTU size supported on inter-partition virtual ethernets is 65280...

After doing some testing ftp'ing a 10G file between 2 machines I've found the with default settings it takes 100 seconds. To set the higher MTU on the card i've done the following:

chdev -l en3 -a mtu=65280 -a tcp_recvspace=655360 -a tcp_sendspace=655360

and the following with no:

no -o tcp_sendspace=655360 -o tcp_recvspace=655360 -o sb_max=1310720

Now my ftp of a 10G file takes 90 seconds, or 10% imporovement. Anything else that needs to be set to get optimum throughput?

 

[p5wizard]

If memory serves me right, the virtual Ethernet theoretical bandwidth in a p5 box is 3Gbit/s, about triple what you're getting, but you probably get slowed down by your disks?

Can you FTP 3 files in parallel from/to different disks? Does that give an overall speed improvement?


HTH,

p5wizard

 

[naabi0]

How do you get 3 time faster?

 

[mrn]

Also make sure your using 100/1000 Full Duplex not autoneg

Mike

"Whenever I dwell for any length of time on my own shortcomings, they gradually begin to seem mild, harmless, rather engaging little things, not at all like the staring defects in other people's characters."