Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations derfloh on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
Replacing a SSA disk in runnig HA/CMP Cluster
- Thread starter haexpert
- Start date
cameltower
IS-IT--Management
i think it's a problem, because if you can change the disk in the array, and you can add the new disk into vg, but i think the standby host must importvg again, so you must stop HACMP.
Replacing SSA disk after it has failed
Basic steps
identify the disk and the resources that use it
remove this disk from any volume groups safely so it is not used anywhere
( this may also mean removing other disks from the configuration aswell )
isolate the disk within the SSA loop
remove the disk definitions
remove and replace the disk
configure the new disk onto the rs6000 this should have the same pdisk and hdisk as the old disk
add to the volume group
add to logical volumes and re-mirror and sync volumes
remove disk definitions from other systems if necessary
configure the new disk onto the rs6000 this should have the same pdisk and hdisk as the old disk
then at a planned time the export and import of the volume group so that the config is up to date, this requires that
the volume group be varied off on the original system, of course to do this it needs to be unused.
1. Check which disk and what it is allocated to it
Normally you have the pdisk number :
Translate pdisk/hdisk ssaxlate –l pdisknnn
Find out volume group lspv | grep hdisknnn
List logical volumes lspv hdisknnn –l
List disks in logical volume lslv lvname –l
List ppartition mapping lslv lvname –m | pg or lslv lvname -m > filename
You also need to know the physical location .. a serial number is useful as the disks have these on the front .. if you have a disk mapping document you maybe able to locate the tray and location from this or use either of the following
diag -- enter -- task selection -- SSA service aids --
link verfication and the the ssa card select the pdisk and hit enter to set identify and note serial number or
set service mode find the pdisk set the service mode and note the serial number
maymap -a -h this will report on all SSA loops and what is on it with the hdisk number and serial numbers
If this is part of a HACMP sysytem or where this disk is connected to multiple systems you need to check the other systems, is the disk in use ... what are the pdisk and hdisk names for it ... use the pvid number to track the disk between systems
2. remove disk from logical volume
run rmlvcopy for each logical volume allocated to that disk and remove all disks for that mirror copy
3. remove disk from volume group
run reducevg command for failed disk on the volume group
4. run diag
use ssa service aids ….. set service mode to identify the pdisk
this will cause the LED onthe front to flash and isolate the disk in the loop ready for hot swap
5. physically replace disk, then reset service aids to ensure service mode is off
6. remove definition of pdisk and hdisk from box and odm
rmdev-dl hdisknnn
rmdev –dl pdisknnn
7. Now configure new disk
cfgmgr
8. Check new disk hdisk number ( hopefully the same as before )
lspv | grep None
9. add to volume group
extendvg vgname hdisknnn
10. now re make logical volume copies
use mklvcopy add 1 more copy to all disk as before for all logical volumes needed
( this sometimes can be complex if a disk is allocated to more than one logical volume then a map file can be used to ensure
that the partition mappings accross disk mirrors are correct, )
The allocation map file contains records specifying the exact physical partitions to allocate to the logical volume. Records are in the form: PVname
Pnum1[-PPnum2], where PVname is the physical volume name and PPnum is the physical partition number. Use one record for each physical partition or each range of consecutive physical partitions. Partitions are used in the order given in the MapFile. Used partitions are skipped.
11. sync logical volumes
syncvg –l lvname
Note : once the logical volume copies are reduced to 1 until the syncvg commands have finished you have no mirroring, if a logical volume uses more that 2 disks in its mirror you increase the risk of data loss through another disk failure once you remove all the mirror disks from a logical volume.
If this is part of a hacmp system then the volume group should be sync'd asap on the other systesm
This requires the original system to have the volume group varried off and an exportvg and importvg run on the other systesm for that volume group
On the fall-over system
Identify the pdisk and hdisk for the failed disk using the pvid of the failed disk ( lspv |grep pvidnumber )
rmdev-dl hdisknnn
rmdev –dl pdisknnn
cfgmgr
ls -la /dev/vgname
rsuk0401:/> ls -la /dev/sapdata1vg
crw-rw---- 1 root system 51, 0 Sep 15 2001 /dev/sapdata1vg
this number in bold is the volume group major number, you will need this latter
lspv | grep vgname
select a hdisk that is not the failed disk
original system
shutdown any application that is using the volume group in question
unmount any filesystems that are from this volume group
df -k
unmount /mountpoint
vary off the volume group
varyoffvg vgname
check volume group is offline
lsvg vgname
if this returns information about the vg then it isn't offline
Fall-over system
exportvg vgname
importvg -yvgname -vmajornumber hdisknumber
this could take several minutes when complete the volume group should be online
lsvg vgname
chvg -an vgname
set the volume group to not auto vary on at boot this is very important with HACMP volume groups
varyoffvg vgname
Original system
varyonvg vgname
mount the filesystems that you unmounted
start the applications back up
Basic steps
identify the disk and the resources that use it
remove this disk from any volume groups safely so it is not used anywhere
( this may also mean removing other disks from the configuration aswell )
isolate the disk within the SSA loop
remove the disk definitions
remove and replace the disk
configure the new disk onto the rs6000 this should have the same pdisk and hdisk as the old disk
add to the volume group
add to logical volumes and re-mirror and sync volumes
remove disk definitions from other systems if necessary
configure the new disk onto the rs6000 this should have the same pdisk and hdisk as the old disk
then at a planned time the export and import of the volume group so that the config is up to date, this requires that
the volume group be varied off on the original system, of course to do this it needs to be unused.
1. Check which disk and what it is allocated to it
Normally you have the pdisk number :
Translate pdisk/hdisk ssaxlate –l pdisknnn
Find out volume group lspv | grep hdisknnn
List logical volumes lspv hdisknnn –l
List disks in logical volume lslv lvname –l
List ppartition mapping lslv lvname –m | pg or lslv lvname -m > filename
You also need to know the physical location .. a serial number is useful as the disks have these on the front .. if you have a disk mapping document you maybe able to locate the tray and location from this or use either of the following
diag -- enter -- task selection -- SSA service aids --
link verfication and the the ssa card select the pdisk and hit enter to set identify and note serial number or
set service mode find the pdisk set the service mode and note the serial number
maymap -a -h this will report on all SSA loops and what is on it with the hdisk number and serial numbers
If this is part of a HACMP sysytem or where this disk is connected to multiple systems you need to check the other systems, is the disk in use ... what are the pdisk and hdisk names for it ... use the pvid number to track the disk between systems
2. remove disk from logical volume
run rmlvcopy for each logical volume allocated to that disk and remove all disks for that mirror copy
3. remove disk from volume group
run reducevg command for failed disk on the volume group
4. run diag
use ssa service aids ….. set service mode to identify the pdisk
this will cause the LED onthe front to flash and isolate the disk in the loop ready for hot swap
5. physically replace disk, then reset service aids to ensure service mode is off
6. remove definition of pdisk and hdisk from box and odm
rmdev-dl hdisknnn
rmdev –dl pdisknnn
7. Now configure new disk
cfgmgr
8. Check new disk hdisk number ( hopefully the same as before )
lspv | grep None
9. add to volume group
extendvg vgname hdisknnn
10. now re make logical volume copies
use mklvcopy add 1 more copy to all disk as before for all logical volumes needed
( this sometimes can be complex if a disk is allocated to more than one logical volume then a map file can be used to ensure
that the partition mappings accross disk mirrors are correct, )
The allocation map file contains records specifying the exact physical partitions to allocate to the logical volume. Records are in the form: PVname
Pnum1[-PPnum2], where PVname is the physical volume name and PPnum is the physical partition number. Use one record for each physical partition or each range of consecutive physical partitions. Partitions are used in the order given in the MapFile. Used partitions are skipped. 11. sync logical volumes
syncvg –l lvname
Note : once the logical volume copies are reduced to 1 until the syncvg commands have finished you have no mirroring, if a logical volume uses more that 2 disks in its mirror you increase the risk of data loss through another disk failure once you remove all the mirror disks from a logical volume.
If this is part of a hacmp system then the volume group should be sync'd asap on the other systesm
This requires the original system to have the volume group varried off and an exportvg and importvg run on the other systesm for that volume group
On the fall-over system
Identify the pdisk and hdisk for the failed disk using the pvid of the failed disk ( lspv |grep pvidnumber )
rmdev-dl hdisknnn
rmdev –dl pdisknnn
cfgmgr
ls -la /dev/vgname
rsuk0401:/> ls -la /dev/sapdata1vg
crw-rw---- 1 root system 51, 0 Sep 15 2001 /dev/sapdata1vg
this number in bold is the volume group major number, you will need this latter
lspv | grep vgname
select a hdisk that is not the failed disk
original system
shutdown any application that is using the volume group in question
unmount any filesystems that are from this volume group
df -k
unmount /mountpoint
vary off the volume group
varyoffvg vgname
check volume group is offline
lsvg vgname
if this returns information about the vg then it isn't offline
Fall-over system
exportvg vgname
importvg -yvgname -vmajornumber hdisknumber
this could take several minutes when complete the volume group should be online
lsvg vgname
chvg -an vgname
set the volume group to not auto vary on at boot this is very important with HACMP volume groups
varyoffvg vgname
Original system
varyonvg vgname
mount the filesystems that you unmounted
start the applications back up
Use C-SPOC to remove definitions of old disk from the system. You will avoid outages for manual re-importing the shared VG on the other cluster nodes.
regards,m.
BTW: If the disk is brand new and have no PVID create it before you run "cfgmgr -l ssar" on cluster nodes.
1. Fist node:
cfgmgr -l ssar
chdev -l hdiskX -pv=yes
2. You can now run "cfgmgr -l ssar" on the rest of cluster nodes.
Doing above you will avoid problems during C-SPOC work with synchronizing resource information.
regards,m.
BTW: If the disk is brand new and have no PVID create it before you run "cfgmgr -l ssar" on cluster nodes.
1. Fist node:
cfgmgr -l ssar
chdev -l hdiskX -pv=yes
2. You can now run "cfgmgr -l ssar" on the rest of cluster nodes.
Doing above you will avoid problems during C-SPOC work with synchronizing resource information.
Read also "Using SMIT to Replace a Cluster Disk", page 128, Redbook SC23-4279-05 (High Availability Cluster Multi-Processing for AIX, Administration Guide)
regards,m.
regards,m.
- Status