FastT600 - Cannot varyonvg on one server, but can varyonvg other serve

[aixolympics]

Good evening,

Thanks a lot for your help, we appreciate it a bunch. Heres basically a summary of what we have (sorry Im adding some extra history as well):

We have 2 Ibm Servers (Hag and Aki) that are running AIX Version 5. Each connect to a SAN device via 2 separate fiber paths. Hag seems to have problems running varyonvg for 3 of the 5 logical volumes, while Aki can varyonvg all of them without a problem. My goal is to get Hag to be able to varyonvg all of the 5 logical volumes. Here is the history:

- Initial Problem (Battery Replacement - Resolved!)
Initially, one of our software GUIs displayed that a battery needed to be replaced in the Fiber adapter on the SAN device. I shutdown the entire system and safely replaced the battery according to the manuals. Upon restarting the system, the LED was green for that device. Upon booting up the servers (Hag and Aki), I noticed...

- Second Problem (Fiber Controller Card Bad, Replaced - Resolved?)
The back of the Hag chassis, has a 2 PCI cards (Fiber Adapters). One of them was bad. I shutdown the machine and replaced the fiber card with a spare. Upon booting it up, the LEDs turned Green like the other Fiber card, so Im assuming that it is connected again. I also used the manual for replacing it (i.e. running `cfgmgr` after replacement). Upon mounting the varyonvg volume groups, I found the current problem...

The SAN has 2 controller cards (controller_A [dac0], controller_B [dac1]) that are used when trying to mount these partitions.

Hag can map ALL virtual partitions if they are mapped through only controller_A (controller_B is disabled). However upon enabling controller_B, some partitions are moved by ADT only controller_A's partitions can be mapped.

Smitty is a centralized administrative application for running all sorts scripts to turn on and off fiber (HBA) devices, hard disks, LVM, and so forth. Do you have any ideas on how I may be able to perform additional tests to switch the mapping? I want to run a command that allows me to test the connectivity to Hag's dac1 (Disk Array Controller).

Here are the outputs of some commands on Hag:
#lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter ***
*** This FC Adapter is the one associated with the fiber connection that was down. (I.e., when I ran this command before replacing the Fiber Adapter on Hag, it was "Defined" instead of "Available").

#lsdev -C | grep dac
dac0 Available 1j-08-01 1722-600 Disk Array Controller
dac1 Available 14-08-01 1722-600 Disk Array Controller

#lsdev -C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

#fget_config -A
--dar0--
dac0 ACTIVE dac1 ACTIVE
hdisk2 dac0
hdisk3 dac0
hdisk4 dac0
--dar1--

*** However on Aki, the results are different since dac1 is used and load distributed (ADT)
#fget_config -A
--dar0--
dac0 ACTIVE dac1 ACTIVE
hdisk2 dac1
hdisk3 dac0
hdisk4 dac1
hdisk5 dac1
hdisk6 dac0
hdisk7 dac1

*** Here are all of the physical volumes:
#lspv #FROM HAG MACHINE (All available vg's are varied on and active).
hdisk0 xxxxxxxxxxxxxxxx rootvg active
hdisk1 xxxxxxxxxxxxxxxx db01vg active
hdisk2 xxxxxxxxxxxxxxxx vg00 active
hdisk3 xxxxxxxxxxxxxxxx vg02 active
hdisk4 xxxxxxxxxxxxxxxx vg03 active
hdisk5 xxxxxxxxxxxxxxxx vg05 active

#lspv #FROM AKI MACHINE
hdisk0 xxxxxxxxxxxxxxxx rootvg active
hdisk1 xxxxxxxxxxxxxxxx None
hdisk2 xxxxxxxxxxxxxxxx vg00
hdisk3 xxxxxxxxxxxxxxxx vg01
hdisk4 xxxxxxxxxxxxxxxx vg02
hdisk5 xxxxxxxxxxxxxxxx vg03
hdisk6 xxxxxxxxxxxxxxxx vg04
hdisk7 xxxxxxxxxxxxxxxx vg05

When I run `varyonvg` on haguroyama, it lets varyonvg: vg00, vg02, vg03. It will not allow me to varyonvg volumes: vg01 (hdisk3), vg04 (hdisk6), and vg05 (hdisk7).

This is all the information I have at the moment. So basically to sum it up:

1) I want fget_config on Hag to have the same hdisks(2-7) as Aki.
2) I want to be able to varyonvg and be load distributed on Hag.

Thank you very much for your help!
Regards,
-Bill

 

[kclute]

Bill,
You stated that you replaced the fc card. Not knowing if you are going through a switch or not, here are a couple things to check.

1) When you replaced the card did you get the new Network Address for the new card (command lscfg -vl fcs1)? And enter that into the mapping for the host port on the Storage Manager for the SAN?

2) If going through a switch you may want to check your zoning.

Kirk

 

[aixolympics]

Hi Kirk, Thanks for your help.

I appreciate it very much. I think we may be onto something here. I located the new WWPN Network Address of the new and old fiber adapter. Im adding them now. I do think we are using Zoning.

Im new to zoning, how do we know for sure if we are using zoning and what needs to be done in the switch for zoning? Is this something I can change by logging into the WEB GUI for each of the switches?

Thank you so very much for your help,
-Bill

 

[aixolympics]

Hi Guys... This is an update.

QUESTION:

How can I get my HAGUROYAMA machine to mount vg00-vg05 volume groups. It cant seem to talk to the SAN, but I made the WPPN access allowed. How can I make it so when I type 'lspv' I see the same output as my working server AKINOUMI. Both are AIX


DETAILS:



Thanks for your help. We appreciate it very much. Today we did a few tests on the machine which may have put Haguroyama closer (or further) away to the working machine Akinoumi.

We noticed that the Haguroyama had 2 dar devices (dar0, dar1) when running your command, while Akinoumi only had 1 dar device (dar0). Thanks for showing us this command, it seemed to have helped resolve the problem a bit. I then decided to use the below steps to make Haguroyama only have 1 dar device:

rmdev -R -dl dar0 #also removed: hdisk2,3,4
rmdev -R -dl dar1 #also removed: hdisk5,6,7
rmdev -dl dac0
rmdev -dl dac1
rmdev -R -dl fcs0
rmdev -R -dl fcs1
cfgmgr -v

After cfgmgr finished running, there was now only 1 dar device, and the mapping seemed to be closer between the two machines. The only difference is two things:

1. Each Hdisk has "none None" as PVid (when viewing with lspv)
2. Each Hdisk on Haguroyama uses the opposite fcs device from Akinoumi.

Im thinking that if we can fix the above 2 problems, we may be able restore the ability for hacmp to varyonvg the volume groups (such as vg00, vg01 and so forth). So here is the question:

"How can we make the `lspv` command produce the same mapping on both Haguroyama and Akinoumi(working)"?

Below are the results of the commands of the current state of the systems.

lspv
getlvodm -C
fget_config -A
lsdev –C | grep dar
lsdev –C | grep dac
lsdev -C | grep fcs
lsdev -Cc disk -F 'name location'
lsdev -Cc driver -F 'name location'


*As you look at the results below you will notice a lot of differences between the machine! Even the fget_config is weird!*
/_______________________\
|Results are as follows:|
\_______________________/

------------------------------------
------------------------------------
-Haguroyama (Not able to varyonvg)--
------------------------------------
------------------------------------

1) lspv
hdisk0 000268fceda1da0 rootvg active
hdisk1 000268fdfe25883 db01vg active
hdisk2 none None
hdisk3 none None
hdisk4 none None
hdisk5 none None
hdisk6 none None
hdisk7 none None

2) getlvodm -C
hdisk0
hdisk1

3) fget_config -A
--- dar0 ---

dac0 ACTIVE dac1 ACTIVE
hdisk2 dac0
hdisk3 dac1
hdisk4 dac0
hdisk5 dac0
hdisk6 dac1
hdisk7 dac1

4) lsdev –C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

5) lsdev –C | grep dac
dac0 Available 1j-08-01 1722-600 (600) Disk Array Controller
dac1 Available 14-08-01 1722-600 (600) Disk Array Controller

6) lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter

7) lsdev -Cc disk -F 'name location'
hdisk0 1A-08-00-8,0
hdisk1 1A-08-01-10,0
hdisk2 1j-08-01
hdisk3 14-08-01
hdisk4 1j-08-01
hdisk5 1j-08-01
hdisk6 14-08-01
hdisk7 14-08-01

8) lsdev -Cc driver -F 'name location'
dar0
fcnet0 1j-08-02
fcnet1 14-08-02
fscsi0 1j-08-01
fscsi1 14-08-01
scsi2 1A-08-00
scsi3 1A-08-01



--------------------------------------------------
______The Other Machine...________________________
--------------------------------------------------


------------------------------------
------------------------------------
---Akinoumi (Works Perfectly)--
------------------------------------
------------------------------------

1) lspv
hdisk0 000b80bfce436320 rootvg active
hdisk1 000b80bf05467e80 None
hdisk2 000b80bf27ceb498 vg00 active
hdisk3 000b80bf6c9b41f8 vg01 active
hdisk4 000b80bf6c9a3a8d vg02 active
hdisk5 000b80bffbb9f2af vg03 active
hdisk6 000b80bffbb6bf7b vg04 active
hdisk7 000b80bf27dc7320 vg05 active

2) getlvodm -C
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
hdisk6
hdisk7

3) fget_config -A
--- dar0 ---

User array name = "Production"
dac0 ACTIVE dac1 ACTIVE

Disk DAC LUN Logical Drive
hdisk2 dac1 1 AIX_1
hdisk3 dac0 2 AIX_2
hdisk4 dac1 3 AIX_3
hdisk5 dac1 4 AIX_4
hdisk6 dac0 5 AIX_5
hdisk7 dac0 6 AIX_6

4) lsdev –C | grep dar
dar0 Available 1722-600 (600) Disk Array Router

5) lsdev –C | grep dac
dac0 Available 1j-08-01 1722-600 (600) Disk Array Controller
dac1 Available 14-08-01 1722-600 (600) Disk Array Controller

6) lsdev -C | grep fcs
fcs0 Available 1j-08 FC Adapter
fcs1 Available 14-08 FC Adapter

7) lsdev -Cc disk -F 'name location'
hdisk0 1A-08-00-8,0
hdisk1 1A-08-01-10,0
hdisk2 14-08-01
hdisk3 1j-08-01
hdisk4 14-08-01
hdisk5 14-08-01
hdisk6 1j-08-01
hdisk7 1j-08-01

8) lsdev -Cc driver -F 'name location'
dar0
fcnet0 1j-08-02
fcnet1 14-08-02
fscsi0 1j-08-01
fscsi1 14-08-01
scsi2 1A-08-00
scsi3 1A-08-01