nfs transfer problem

[Surfeur]

Hi, all.
I have a REALLY problem with nfs server/client. Topology: 4 nodes, 2 clusters; on one node is exported directory /ap/fh/common (this node gena04 is server and client). It's mounted over all nodes (source including) like /ap/fh/common_gena04. When I use ftp or cp command for any file from this node or other node to /ap/fh/common the transfer speed is ok. But when I made the same for nfs mount, the speed is very low and the transfer is splited (e.g. 1 MB is transferred and for next 30 sec nothing more). When some refresh of file is made on this node it caused stopping of communication some times for 2 minutes (updted files are abou 200 bytes). Client is setup ok.

Thanks for whatever idea. Vladimir

/etc/exports
============================================================
/ap/fh/common -root=gena03b:gena04b:gena05b:gena06b:gena07b:gena08b:gena03:gena04:gena05:gena06:gena07:gena08

gena04:

Server rpc:
Connection oriented
calls badcalls nullrecv badlen xdrcall dupchecks dupreqs
76256675 0 0 0 0 28239165 0
Connectionless
calls badcalls nullrecv badlen xdrcall dupchecks dupreqs
112736117 0 0 0 0 42629439 241258

Server nfs:
calls badcalls public_v2 public_v3
141314418 238596 0 0
Version 2: (13448 calls)
null getattr setattr root lookup readlink read
0 0% 1911 14% 0 0% 0 0% 4256 31% 0 0% 7261 53%
wrcache write create remove rename link symlink
0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0%
mkdir rmdir readdir statfs
0 0% 0 0% 11 0% 9 0%
Version 3: (141300970 calls)
null getattr setattr lookup access readlink read
85 0% 34487220 24% 14007799 9% 8582055 6% 8844137 6% 0 0% 4428804 3%
write create mkdir symlink mknod remove rmdir
14010425 9% 1983 0% 0 0% 0 0% 0 0% 1682 0% 0 0%
rename link readdir readdir+ fsstat fsinfo pathconf
74 0% 0 0% 22392 0% 42846129 30% 50682 0% 85 0% 0 0%
commit
14017418 9%

Client rpc:
Connection oriented
calls badcalls badxids timeouts newcreds badverfs timers
24007389 0 0 0 0 0 0
nomem cantconn interrupts
0 0 0
Connectionless
calls badcalls retrans badxids timeouts newcreds badverfs
38 0 0 0 0 0 0
timers nomem cantsend
0 0 0

Client nfs:
calls badcalls clgets cltoomany
24004867 0 0 0
Version 2: (0 calls)
null getattr setattr root lookup readlink read
0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0%
wrcache write create remove rename link symlink
0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0%
mkdir rmdir readdir statfs
0 0% 0 0% 0 0% 0 0%
Version 3: (24006785 calls)
null getattr setattr lookup access readlink read
0 0% 10174183 42% 4278709 17% 508405 2% 237519 0% 0 0% 224092 0%
write create mkdir symlink mknod remove rmdir
4285504 17% 1 0% 0 0% 0 0% 0 0% 1 0% 0 0%
rename link readdir readdir+ fsstat fsinfo pathconf
0 0% 0 0% 0 0% 2913 0% 17065 0% 4 0% 1 0%
commit
4278388 17%

 

[Yegolev]

first impression is to check DNS and /etc/hosts on all machines, then verify correct names in /etc/exports.

IBM Certified -- AIX 4.3 Obfuscation

 

[madmurdock]

Hmmm, that could be a long search? When yoiu talk about nodes, do you mean SP-nodes with a cws and they talk over a 10MBIT Twisted Pair? Or is this a switched connection ( that means with a switch in the middle) then check Speed on cable (10 or 100 MBIT) and Duplexing ( HALF or FULL )

Another suggestiuon is to check the MTU-sizes on all machines, and of course, name-resolution !

Hope that helps

mad_murdock

 

[Surfeur]

More detailed:
=============
6 machines pseries660, SSA disks
Ethernet 100Mb en0 en1 en3
Ethernet card:
gena04:# lsattr -El ent0
busmem 0xc0030000 Bus memory address False
rom_mem 0xc0000000 ROM memory address False
busintr 35 Bus interrupt level False
intr_priority 3 Interrupt priority False
txdesc_que_sz 512 TX Descriptor Queue Size True
rxdesc_que_sz 512 RX Descriptor Queue Size True
tx_que_sz 8192 Software TX Queue Size True
rxbuf_pool_sz 1024 Receive Buffer Pool Size True
media_speed Auto_Negotiation Media Speed True
use_alt_addr no Enable Alternate Ethernet Address True
alt_addr 0x400000000004 Alternate Ethernet Address True
poll_link no Enable Link Polling True
poll_link_timer 500 Time interval for Link Polling True

Topology:

gena03 gena04
gena05===Routers============Routers======gena06
-rack- cca 1 km -rack-
gena07 gena08

hosts file is generated for all nodes from one source over all nodes. There are used only static IP adresses. No DNS.

 

[Yegolev]

change your adapters to ANYTHING aside from autonegotiate. best if everyone uses same speed but definitely don't use autonegotiate.

IBM Certified -- AIX 4.3 Obfuscation

 

[nabob1]

Change your NIC speed to 100 Full Duplex

 

[Surfeur]

Thanx for all answers. I have made the ethernet adapter changes on all nodes to 100_full_duplex. I have made the changes in nfs deamons: nfsd(server) up to 4.500 and biod (clients) up to 500. It looks better, but make better performance, but not far all time. On the test server i tried to use CacheFs for nfs. An it work ok. But do you have anybody expiriencies with it in HACMP. Mainly in case of takeover. I'll try it, some time nfskill in event is not working properrlly with CacheFs directory.

Greetings
Vladimir