UPS solution for P5 570 2

[khalidaaa]

Hi all,

I've been asked to implement a UPS solution to our two production P5 570s machines. Now they are both connected to a UPS but there is no RS232 cable between them which means that the UPS is acting as a power equalizer but of no use when power goes off because the LPARs within the P5 machine won't know about it until it discharge.

Now i'm thinking of more than one senario! I can connect the RS232 to one of the LPARs which will be triggered by the power outage and will remotely issue a shutdown to the rest of the LPARs and then shutdown itself (then handing over the task to the HMC to shutdown the P5 racks and finally shutdown itself). But the problem with this senario is that the network switches are not on UPS! so i can't use the external network for this! I can do vertual network and communicate internally among LPARs on the same box but the problem with that is that we still have AIX 5.2 LPARs which doesn't support this and i won't be able to communicate with the HMC to shutdown the P5s!

Is there a way that i can use the HMC internal network for this? I'm not sure how the HMC is cabeled with the Rack of the two P5 machines so i don't have the clear picture to be honest on that! But i feel that there should logically be possible to communicate with the HMC using the internal network (from one LPAR that has the RS232 connection to the UPS).

I had this site but still it doesn't say that i can communicate between the LPARs and the HMC using the interal HMC network (as i beleive this is only connecting the FSP or so):

HMC connections
During the installation of the HMC, you have to consider the number of network adapters required. You can have up to three Ethernet adapters installed on an HMC. There are several connections you have to consider when planning the installation of the HMC: 

HMC to the FSP (Flexible Service Processor): It is an IP-based network used for management functions of the POWER5 systems; for example, power management and partition management.
Note: It is recommended to configure this connection as a private network. 
POWER5 systems have two interfaces (T1 and T2) available for connections to the HMC. It is recommended to use both of them for redundant configuration, and high availability. Depending on your environment, you have multiple options to configure the network between the HMC and FSP. The default mechanism for allocation of the IP addresses for the FSP ports is dynamic. The HMC can be configured as a DHCP server which allocates the IP address at the time the managed system is powered on. Static IP address allocation is also an option. You can configure the FSP ports with a static IP address by using the Advanced System Management Interface (ASMI) menus. However not all POWER5 servers support this mechanism of allocation. Currently p575, p590, and p595 servers support only DHCP. 
Note: Either eth0 or eth1 can be a DHCP server on the HMC. 
HMC to partitions: HMC requires TCP/IP connection to communicate with the partitions for functions such as dynamic LPAR and Service Focal Point. 
Service Agent (SA) connections: SA is the application running on the HMC for reporting hardware failures to the IBM support center. It uses a modem for dial-out connection or an available Internet connection. It can also be used to transmit service and performance information to IBM and also for CoD enablement and billing information. 
Remote connection to the HMC using Web-based System Manager (WebSM) or ssh: For accessing the graphical interface, you can use the WebSM Remote Client running on UNIX® (AIX® or Linux®) or Windows®. The command line interface is also available by using the secure shell connection to the HMC. It can be used by an external management system or a partition to perform HMC operations remotely.
When planning for the HMC installation also consider that the distance between the HMC and the managed system must be within 8m (26 ft) distance. The distance complies with IBM maintenance rules.

http://www-941.ibm.com/collaboration/wiki/display/WikiPtype/HMC+attached+System+Setup

Could you please share with me your experience in this? how do you maintain your machines on power outage?

Thanks in advance.

Regards,
Khalid

 

[p5wizard]

I'd put the network switches (at least the ones that involve the p5 LPARs and the HMC) also on the UPS.

FSP to HMC network is just that. It links the HMC to its managed servers but you can't use it for anything else.


HTH,

p5wizard

 

[khalidaaa]

I think (only think coz i'm not sure about this) that the P5 LPARs connect to the HMC through our core switches and unfortunately they are not on UPS.

I don't want to recommend having the UPS on the core switches unless i'm sure that i can't do this internally among the two P5 machines along with the HMC!

There is a small hub that is used ot connect the HMC with the 2 P5 570s but i'm not sure what are those connections to be honest!

The hub contains 8 ports and here is the document i got on that from the IBM engineer:

Port 1: To LAN Network
Port 2: (Integrated) HMC Connection to External Network
Port 3: (PCI slot) HMC Connection to Internal Network
Port 4: P5 RACK1 (SYS1)
Port 5: Free (Can be used to connect to SAN Switch
Port 6: P5 RACK2 (SYS2)
Port 7: P5 SAN1
Port 8: P5 SAN2

Can you please explain to me this?

Thanks P5wizard

Regards,
Khalid

 

[p5wizard]

I have no idea... it doesn't make sense to me to have both internal and external HMC connection on the same network hub or switch??? Perhaps you need to clarify this with the IBM engineer?

A question though: why have a UPS on the p5 machines when in case of power failure your whole network goes down?


HTH,

p5wizard

 

[khalidaaa]

Thanks P5wizard for your inputs!

You are right about having the network going down which doesn't make sense. But currently i'm doing a feasibility study to whether buy a UPS for the network or just connect an RS232 to all servers (sense all the servers are connected to UPS) to trigger the power shutdown for them (which in this case i won't need a UPS on the network).

It's complicated but i have to choose the best solution!

Any other thoughts?

Regards,
Khalid

 

[p5wizard]

if HMC is also on UPS, why don't you run a ssh script from one LPAR to the HMC to issue shutdown on all LPARs via HMC CLI command chsysstate?

from man page:[tt]
To shut down or restart a partition:
chsysstate -m managed-system -r lpar
-o {shutdown | osshutdown | dumprestart |
retrydump}
{-n partition-name | --id partition-ID}
[--immed] [--restart]

[/tt]

loop around the managed systems and their lpars and shut 'em down (note I'm just typing this here, it's untested):

[tt]for sys in $(lssyscfg -r sys -F name)
do
for lpar in $(lsyscfg -r lpar -m ${sys} -F name,state|grep Running|cut -d, -f1)
do
chsysstate -m ${sys} -r ${lpar} -o osshutdown --immed
done
done[/tt]

You'll wan't to fine tune this, so that all VIO clients are shut down before the VIO servers are halted, so:
[tt]
for lpar in $(lsyscfg -r lpar -m ${sys} -F lpar_env,name,state | sort | grep Running |cut -d, -f2)
[/tt]

And of course no app is first shut down by this script...


HTH,

p5wizard

 

[khalidaaa]

Thanks P5wizard for your input. You are right about the chsysstate as i had it as an option but what i'm not sure of is whether it is possible to talk from the LPAR to the HMC without using the network (or only using the hub that connects the HMC with the other two racks)!

You just said that i can't use the cable that connects the HMC with the FSP right? So would this work using the other cables shown in the above reply?

Regards,
Khalid

 

[p5wizard]

I can't tell from where I'm sitting... As I said before it is kind of strange to have all HMC network cables running to the same hub...

But maybe you have an unused network port on your HMC and an unused ethernet adapter in a PCI slot that you can allocate to your UPS-connected LPAR, why not set up a private ethernet connection with a UTP-cross cable between them? That way you won't have to worry about which network switch is UPS-backed or not...


HTH,

p5wizard

 

[khalidaaa]

hmmm,

Never thought of this! Unfortunately i've just checked and there is only one ethernet connecting the HMC with the hub!

You have been very helpful so far p5wizard so have a star for that.

Any other thoughts?

Regards,
Khalid

 

[p5wizard]

Well, look at the back of the HMC to see if a free ethernet port is available. Or go into "Change Network Settings" on "HMC Management" and see if a LAN adapter port is still unused.


HTH,

p5wizard

 

[khalidaaa]

That's exactly what i did! i looked into the back of the HMC! but found only one adapter!

 

[abubasim]

Salaams Khalid,

Most recent HMCs I've come across have had two Ethernet ports and having only one doesn't make much sense as two is minimum for full functionality: internal for FSP connection (console and LPAR power on/off) and external for DLPAR. According to the config for your 7310-C04 HMC, it should have a gigabit port on the motherboard plus an extra PCI Ethernet card.

What brand of UPS do you have? Seems kind of old-fashioned if it only has a serial port. Check with the UPS vendor if they also offer LAN attachment. One popular UPS brand here in Oman, the MGE Comet, can be equipped with an RJ45 Ethernet port and they provide a small UPS agent for AIX and other platforms which can be used for monitoring and signaling of shutdown.

 

[p5wizard]

Nowadays HMCs have one eth port on mainboard and a dual eth PCI adapter...

And you can always add an additional eth PCI adapter. But that messes up the LAN configuration in RedHat though (numbering ogfeth LAN adapters is suddenly all wrong).


HTH,

p5wizard

 

[khalidaaa]

Salaams Abubasim

Abubasim and p5wizard, i think you are right, i just got another check on the back of the HMC and i found the other PCI slot! i was only looking at the other one. But both of them are connected! as i just described above

Port 2: (Integrated) HMC Connection to External Network
Port 3: (PCI slot) HMC Connection to Internal Network
Port 4: P5 RACK1 (SYS1)
Port 5: Free (Can be used to connect to SAN Switch
Port 6: P5 RACK2 (SYS2)

So port2 is connecting the integrated HMC ethernet and port3 is connecting the PCI HMC ethernet! (I still don't know why we have both of them and the others - i.e. port4 and 5?)

You are right abubasim when you said that's an old fashion UPS coz it has been there for a very long time in my compnay! and it only has an RS232! no ethernet ports!

My worry is not the UPS, my worry is can i communicate using the internal HMC network to the other LPARs when the power goes off? currently i use the main switches (thru the public network).

Have a star Abubasim for the great contribution

Regards,
Khalid