oanacristina
Technical User
Beginning from last week we began to receive in the /tmp/hacmp.out file this sequence of messages:
Nov 24 00:22:58 EVENT START: fail_standby aixapp 10.10.10.10
-------------------
fail_standby[438] [[ high = high ]]
fail_standby[438] version=1.8
fail_standby[439] fail_standby[439] cl_get_path
HA_DIR=
fail_standby[441] NODENAME=aixapp
fail_standby[442] ADDR=10.10.10.10
fail_standby[444] [ 2 -ne 2 ]
fail_standby[450] set -u
fail_standby[452] fail_standby[452] dspmsg scripts.cat 335 Adapter 10.10.10.10 is no longer available for use as a standby,\n due to either a standby adapter failure or IP address takeover.\n 10.10.10.10
MSG=Adapter 10.10.10.10 is no longer available for use as a standby,
due to either a standby adapter failure or IP address takeover.
fail_standby[453] echo Adapter 10.10.10.10 is no longer available for use as a standby, due to either a standby adapter failure or IP address takeover.
fail_standby[453] 1> /dev/console
fail_standby[457] addback_standby_route 10.10.10.10
fail_standby[3] set -u
fail_standby[5] down_if=10.10.10.10
fail_standby[7] read a b NETWORK c
fail_standby[7] cllsif -Sn 10.10.10.10
fail_standby[9] morestandby=0
fail_standby[13] IFS=:
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] cllsif -Sc
fail_standby[14] [[ xxx.yyy.12.22 = 10.10.10.10 ]]
fail_standby[14] [[ boot != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.20 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ 10.10.10.10 = 10.10.10.10 ]]
fail_standby[16] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ /dev/tmscsi0 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.23 = 10.10.10.10 ]]
fail_standby[14] [[ boot != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.21 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ 10.10.10.20 = 10.10.10.10 ]]
fail_standby[14] [[ standby != standby ]]
fail_standby[14] [[ ether_pub != ether_pub ]]
fail_standby[14] [[ aixora != aixapp ]]
fail_standby[22] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ /dev/tmscsi0 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[50] unset IFS
fail_standby[50] [[ 0 -eq 0 ]]
fail_standby[52] return 0
fail_standby[459] exit 0
Nov 24 00:22:58 EVENT COMPLETED: fail_standby aixapp 10.10.10.10
Nov 24 00:23:35 EVENT START: join_standby aixapp 10.10.10.10
join_standby[107] [[ high = high ]]
join_standby[107] version=1.8
join_standby[108] join_standby[108] cl_get_path
HA_DIR=
join_standby[110] NODENAME=aixapp
join_standby[111] ADDR=10.10.10.10
join_standby[113] [ 2 -ne 2 ]
join_standby[119] set -u
join_standby[121] join_standby[121] dspmsg scripts.cat 330 Standby adapter 10.10.10.10 is now available.\n 10.10.10.10
MSG=Standby adapter 10.10.10.10 is now available.
join_standby[122] echo Standby adapter 10.10.10.10 is now available.
join_standby[122] 1> /dev/console
join_standby[125] getroute_of_down_standby_adapters 10.10.10.10
join_standby[2] set -u
join_standby[4] join_if=10.10.10.10
join_standby[5] unset IFS
join_standby[6] read a b NETWORK c
join_standby[6] cllsif -Sn 10.10.10.10
join_standby[7] IFS=:
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] cllsif -Sc
join_standby[8] [[ xxx.yyy.12.22 = 10.10.10.10 ]]
join_standby[8] [[ boot != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.20 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ 10.10.10.10 = 10.10.10.10 ]]
join_standby[9] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ /dev/tmscsi0 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.23 = 10.10.10.10 ]]
join_standby[8] [[ boot != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.21 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ 10.10.10.20 = 10.10.10.10 ]]
join_standby[8] [[ standby != standby ]]
join_standby[8] [[ ether_pub != ether_pub ]]
join_standby[8] [[ aixora != aixapp ]]
join_standby[12] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ /dev/tmscsi0 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[30] unset IFS
join_standby[31] return 0
join_standby[126] exit 0
Nov 24 00:23:35 EVENT COMPLETED: join_standby aixapp 10.10.10.10
-------------
I mention that we have 2 servers IBM RS 6000, aixapp where is installed the Internet application server and aixora where is located the Oracle Database. The operating system is AIX 5.1.
We received such sequence of messages last Thursday when the server blocked and for a while (about 15 minutes) we couldn't connect to the server (aixora). After that period of time it seemed that things came back to normal automatically.
What could be the cause? What is happening? Can you give us an idea?
Nov 24 00:22:58 EVENT START: fail_standby aixapp 10.10.10.10
-------------------
fail_standby[438] [[ high = high ]]
fail_standby[438] version=1.8
fail_standby[439] fail_standby[439] cl_get_path
HA_DIR=
fail_standby[441] NODENAME=aixapp
fail_standby[442] ADDR=10.10.10.10
fail_standby[444] [ 2 -ne 2 ]
fail_standby[450] set -u
fail_standby[452] fail_standby[452] dspmsg scripts.cat 335 Adapter 10.10.10.10 is no longer available for use as a standby,\n due to either a standby adapter failure or IP address takeover.\n 10.10.10.10
MSG=Adapter 10.10.10.10 is no longer available for use as a standby,
due to either a standby adapter failure or IP address takeover.
fail_standby[453] echo Adapter 10.10.10.10 is no longer available for use as a standby, due to either a standby adapter failure or IP address takeover.
fail_standby[453] 1> /dev/console
fail_standby[457] addback_standby_route 10.10.10.10
fail_standby[3] set -u
fail_standby[5] down_if=10.10.10.10
fail_standby[7] read a b NETWORK c
fail_standby[7] cllsif -Sn 10.10.10.10
fail_standby[9] morestandby=0
fail_standby[13] IFS=:
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] cllsif -Sc
fail_standby[14] [[ xxx.yyy.12.22 = 10.10.10.10 ]]
fail_standby[14] [[ boot != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.20 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ 10.10.10.10 = 10.10.10.10 ]]
fail_standby[16] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ /dev/tmscsi0 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.23 = 10.10.10.10 ]]
fail_standby[14] [[ boot != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ xxx.yyy.12.21 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ 10.10.10.20 = 10.10.10.10 ]]
fail_standby[14] [[ standby != standby ]]
fail_standby[14] [[ ether_pub != ether_pub ]]
fail_standby[14] [[ aixora != aixapp ]]
fail_standby[22] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[14] [[ /dev/tmscsi0 = 10.10.10.10 ]]
fail_standby[14] [[ service != standby ]]
fail_standby[18] continue
fail_standby[14] read adapt type network net_type attrib node ip_addr hw_addr
fail_standby[50] unset IFS
fail_standby[50] [[ 0 -eq 0 ]]
fail_standby[52] return 0
fail_standby[459] exit 0
Nov 24 00:22:58 EVENT COMPLETED: fail_standby aixapp 10.10.10.10
Nov 24 00:23:35 EVENT START: join_standby aixapp 10.10.10.10
join_standby[107] [[ high = high ]]
join_standby[107] version=1.8
join_standby[108] join_standby[108] cl_get_path
HA_DIR=
join_standby[110] NODENAME=aixapp
join_standby[111] ADDR=10.10.10.10
join_standby[113] [ 2 -ne 2 ]
join_standby[119] set -u
join_standby[121] join_standby[121] dspmsg scripts.cat 330 Standby adapter 10.10.10.10 is now available.\n 10.10.10.10
MSG=Standby adapter 10.10.10.10 is now available.
join_standby[122] echo Standby adapter 10.10.10.10 is now available.
join_standby[122] 1> /dev/console
join_standby[125] getroute_of_down_standby_adapters 10.10.10.10
join_standby[2] set -u
join_standby[4] join_if=10.10.10.10
join_standby[5] unset IFS
join_standby[6] read a b NETWORK c
join_standby[6] cllsif -Sn 10.10.10.10
join_standby[7] IFS=:
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] cllsif -Sc
join_standby[8] [[ xxx.yyy.12.22 = 10.10.10.10 ]]
join_standby[8] [[ boot != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.20 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ 10.10.10.10 = 10.10.10.10 ]]
join_standby[9] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ /dev/tmscsi0 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.23 = 10.10.10.10 ]]
join_standby[8] [[ boot != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ xxx.yyy.12.21 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ 10.10.10.20 = 10.10.10.10 ]]
join_standby[8] [[ standby != standby ]]
join_standby[8] [[ ether_pub != ether_pub ]]
join_standby[8] [[ aixora != aixapp ]]
join_standby[12] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[8] [[ /dev/tmscsi0 = 10.10.10.10 ]]
join_standby[8] [[ service != standby ]]
join_standby[10] continue
join_standby[8] read adapt type network net_type attrib node ip_addr hw_ddr
join_standby[30] unset IFS
join_standby[31] return 0
join_standby[126] exit 0
Nov 24 00:23:35 EVENT COMPLETED: join_standby aixapp 10.10.10.10
-------------
I mention that we have 2 servers IBM RS 6000, aixapp where is installed the Internet application server and aixora where is located the Oracle Database. The operating system is AIX 5.1.
We received such sequence of messages last Thursday when the server blocked and for a while (about 15 minutes) we couldn't connect to the server (aixora). After that period of time it seemed that things came back to normal automatically.
What could be the cause? What is happening? Can you give us an idea?