Ethernet interface problem

[Smovk]

I`m new to Sun.
We have Sun Sollaris 10 on netra 210 with two interfaces, one is connected to Avaya PBX (link is fixed to 10Mbps HalfDuplex, because this hardware on Avaya PBX is old card and it does not support any other speed) and second interface is connected to LAN.
Till couple of days ago everything was working fine and without any errors in log files. Couple of days ago someone accidentally turned power off for all our equipment.
When system had rebooted, interface bge0 on Sun is getting down and back up every few hour.

this is what we have in /var/adm/messages
Nov 24 00:20:04 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link down
Nov 24 00:20:24 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link up 10
Mbps Half-Duplex
Nov 24 01:20:15 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link down
Nov 24 01:20:35 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link up 10
Mbps Half-Duplex
Nov 24 02:49:25 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link down
Nov 24 02:49:45 0842TFI050 bge: [ID 801593 kern.notice] NOTICE: bge0: link up 10
Mbps Half-Duplex

can you help, I do not know what is wrong. I change cables, and also equipment on Avaya PBX, and every time is the same result

 

[elgrandeperro]

how did you fix the speed on the Solaris side?

 

[Smovk]

it is fixed to 10Mbps half duplex

 

[domburns62]

*How* did you set the speed to 10/half?

Did you use a line in /etc/system or a startup script? Or change a line in an driver config file? These would be a setting that would take effect on reboot.

If it was it done using ndd at the command line. and not in a boot time script or setting, then the setting is lost.

If it was set in a boot item I would check wherever it was set to make sure it wasn't changed sometime, which would take effect after the reboot.

The errors above are indications of the connection being renegotiated.

It could be a flaky hardware part on the Sun side, but the symptoms (started to fail after a reboot) are those of a lost or changed config setting.

 

[elgrandeperro]

I think bge can only be set via editing bge.conf or using a ndd script..I think. I don't think the bge driver uses /etc/system.

 

[Smovk]

I set network speed using ndd command. After reboot speed was still set to 10Mbps Half duplex

this is after rebooting Sun server:

0842TFI050# ifconfig -a
lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
ether 0:21:28:3:53:24
bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 10.0.21.14 netmask ffffff00 broadcast 10.0.21.255
ether 0:21:28:3:53:25

0842TFI050# ndd /dev/bge0 link_status
1
0842TFI050# ndd /dev/bge0 link_speed
10
0842TFI050# ndd /dev/bge0 link_duplex
1
0842TFI050# ndd /dev/bge0 link_autoneg
0

 

[elgrandeperro]

ndd (without being in a startup script) is not persistent.
put the commands in a startup script.

 

[Smovk]

I know thad ndd command is not persistent. Sun is recomending that if you have bge interface that driver should be configured in /platform/sun4u/kernel/drv/bge.conf and not in the startup script.
I found that I made mistake with setting up speed using command ndd /dev/bge0 link_autoneg 0. I found in sun documents that if I have bge interface I should use ndd /dev/bge0 adv_link_autoneg 0 command instead.
When I used /dev/bge0 adv_link_autoneg 0 then my interface bge0 went down and it did not come up.
iWhen I used ifconfig -a command for interface bge0 it showh that is up, but when I tried ping there was no response and in /var/adm/messages it showed that bge0 is initializing.
I?m 10000% convinced that the PBX side is set to 10Mbps halfduplex, because there is no other possible speed except 10mbps half, hardware does not let me change it to auto or any other speed.
Does any one have any idea what is wrong?

 

[Annihilannic]

Slightly OT, but I thought you had to use ndd like this:

ndd -set /dev/bge instance 0   # the default anyway, but...
ndd -set /dev/bge adv_link_autoneg 0

Has it been updated in Solaris 10 to allow specifying an individual device instance like /dev/bge0?

Annihilannic.

 

[Smovk]

During this weekend I changed hardware on PBX so that it is capable 100Mbps Fullduplex. When I fixed 100 Mbps FD on PBX site and on Sun using command:
ndd -set /dev/bge0 adv_100fdx_cap 1
ndd -set /dev/bge0 adv_100hdx_cap 0
ndd -set /dev/bge0 adv_10fdx_cap 0
ndd -set /dev/bge0 adv_10hdx_cap 0
ndd -set /dev/bge0 adv_autoneg_cap 0
then I did not had any Ethernet connection between.
When I set Sun server back to autonegotiation and PBX site to 100Mbps FullDuplex, Sun server connected to 100Mbps Half Duplex. When I set both site to autonegotiation, both sites were connected with 100 Mbps FullDuplex.

This two boxes are directly connected with cable without any switch between.
Any idea why is this happening?

 

[elgrandeperro]

Did you zero out adv_1000fdx_cap and adv_1000hdx_cap?

 

[Smovk]

yes, i just to forgot to write it in the post

 

[elgrandeperro]

this is stupid, but what kind of cable are you using? (straight or rolled, catx)?

 

[elgrandeperro]

Also, if you have a programmable switch to stick between to see how each side negotiates. Or even one side at a time, as long as the switch is set to the right speed and dupex, since you are forcing.

 

[Smovk]

I tried both cable (straight & rolled, cat5) with same result.

 

[elgrandeperro]

I think we can safely rule out Solaris, did you set the Avaya speed/duplex? Do you have a managed switch you can plug it into? ( it doesn't have to work, we just want to see how it negotiates.

 

[Smovk]

I change the switch between PBX and SUN, so SUN server(running Avaya CMS) and C-LAN (PBX side) are now connected with forced 100Mbps Fulduplex.
Now Ethernet link is stable, on C-Lan I do not have any errors (everything is empty for active errors and for cleard errors - I used display error command in SAT), there is no error in Solaris error log /var/adm/messages for interface bge0 going down and back up. In CMS error log under maintenance menu I still have error code 1400 SPI session errorata collection session is down.

I found error in spi_err_1 log (in attached file). In attached file are file spi_err_1 and /var/adm/messages. I could not found any errors in /var/adm/messages for the time when error occurred in spi_err_1 log.