Solaris 9 on v880 boot issue

[vesposito]

Has anyone seen this issue before?

I have a v880 which will not boot after an OS recovery, the system hangs partway through boot. I've booted the system with a -rv option so I can get some idea where it is hanging- the last lines of output are:

cpu7: UltraSPARC-III+ (portid 7 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu0: UltraSPARC-III+ (portid 0 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu1: UltraSPARC-III+ (portid 1 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu2: UltraSPARC-III+ (portid 2 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu3: UltraSPARC-III+ (portid 3 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu4: UltraSPARC-III+ (portid 4 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu5: UltraSPARC-III+ (portid 5 impl 0x15 ver 0xb0 clock 1200 MHz)
cpu6: UltraSPARC-III+ (portid 6 impl 0x15 ver 0xb0 clock 1200 MHz)

The potential sore spot is that the recovered OS image is actually from a v480, we had to recover to an 880 due to hardware restrictions.

We've taken the steps you'd expect- rebuilding path_to_inst rebuilding /dev and /devices, commenting references to vxvm from /etc/system, and using a vfstab appropriate for the new hardware.

Vincent Esposito

 

[Annihilannic]

Do the v480 and v880 have identical architectures?

Annihilannic.

 

[tuka]

These two system have different architecture.

Sun Fire[TM] v480 / Sun Fire[TM] v490

PCI Slot 0 /pci@8,600000/<device>@1,* 33/66mhz-32/64bit-3.3V
PCI Slot 1 /pci@8,600000/<device>@2,* 33/66mhz-32/64bit-3.3V
PCI Slot 2 /pci@8,700000/<device>@2,* 33mhz-32/64bit-5V
PCI Slot 3 /pci@8,700000/<device>@3,* 33mhz-32/64bit-5V
PCI Slot 4 /pci@8,700000/<device>@4,* 33mhz-32/64bit-5V
PCI Slot 5 /pci@8,700000/<device>@5,* 33mhz-32/64bit-5V
Disk 1 /pci@9,600000/SUNW,qlc@2/fp@0,0
Disk 2 /pci@9,600000/SUNW,qlc@2/fp@1,0
DVD/CDROM /pci@8,700000/ide@6
Network 1 /pci@9,600000/network@1
Network 2 /pci@9,700000/network@2
EBUS /pci@9,700000/ebus@1
USB /pci@9,700000/usb@1,3



Sun Fire[TM] v880

PCI Slot 0 /pci@8,700000/<device>@5,* 33mhz-32/64bit-5V
PCI Slot 1 /pci@8,700000/<device>@4,* 33mhz-32/64bit-5V
PCI Slot 2 /pci@8,700000/<device>@3,* 33mhz-32/64bit-5V
PCI Slot 3 /pci@8,700000/<device>@2,* 33mhz-32/64bit-5V
PCI Slot 4 /pci@9,700000/<device>@4,* 33mhz-32/64bit-5V
PCI Slot 5 /pci@9,700000/<device>@3,* 33mhz-32/64bit-5V
PCI Slot 6 /pci@9,700000/<device>@2,* 33mhz-32/64bit-5V
PCI Slot 7 /pci@9,600000/<device>@2,* 33/66mhz-32/64bit-3.3V
PCI Slot 8 /pci@9,600000/<device>@1,* 33/66mhz-32/64bit-3.3V
DVD/ CDROM /pci@8,700000/scsi@1/sd@6,0
Onboard FCAL-loopA /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@<WWN>,0
loopB* /pci@9,600000/pci@1/SUNW,qlc@4/fp@0,0/ssd@<WWN>,0
EBUS /pci@9,700000/ebus@1/<device>
Network 1 (ge) /pci@8,600000/network@1
Network 2 (eri) /pci@9,700000/network@1,1
USB /pci@9,700000/usb@1,3/<device>

 

[Annihilannic]

By architecture I'm referring to the output of uname -i and uname -m.

If they are different you will probably need to install a different boot block.

Annihilannic.

 

[vesposito]

We were able to correct the problem with little issue after some research into areas not "officially" supported by sun.

The bootblock was not the issue, we did install a valid bootblock after the restore, however in rebuilding device paths etc, our client initially missed rebuilding /etc/name_to_major. Don't bother calling sun if you see an issue like this in the same situation- they will tell you it cannot be done, however, below is the routine we used sucessfully. To start, we booted off of a clean OS, mounted the second disk as /mnt and restored the client's OS there, then installed a new bootblock- we then ran with the below- note you can do the same thing if you boot off cdrom first and mount your new root disk:

1) Back up some of the files we will be modifying:
cd /mnt/etc
cp -p path_to_inst path_to_inst.old
cp -p vfstab vfstab.old
cp -p name_to_major name_to_major.old

2) move aside /dev/dsk and /dev/rdsk
cd /mnt/dev
mv dsk dsk.old
mv rdsk rdsk.old

3) bring over /dev/dsk and /dev/rdsk from your valid running system (primary clean install or cdrom)
cd /dev
tar cf - ./dsk | (cd /a/dev; tar xvfp -)
tar cf - ./rdsk | (cd /a/dev; tar xvfp -)

4) reconfigure device paths/path_to_inst
cd /mnt/devices
drvconfig -r . -p ../etc/path_to_inst
cd /mnt
disks -r /mnt
ports -r /mnt
devlinks -r /mnt
tapes -r /mnt
audlinks -r /mnt

5) copy over /etc/name_to_major from valide live system or cdrom miniroot
cp /etc/name_to_major /mnt/etc/name_to/major

6) Modify /mnt/etc/vfstab to reflect the filesystem layout of the recovered system

7) Cleanup items- you may need to go into /mnt/etc/system and remove references for vxvm/vxfs if you're using them- and possipbly /mnt/etc/rc.* startup scripts for veritas if you don't plan on using them immediately- you're better off reinstalling/reconfiguring veritas after bringing up the recovered system than trying to get the recovered install working.

8) more cleanup
umount /mnt
You may not need to, but best practices are to fsck the recovered root disk just to be sure-
fsck -y /dev/rdsk/c*t*d*s*

9) init 0 down to ok prompt (or sync; sync; halt if you're old-school)

10) boot the recovered root:
boot (device path to recovered root) -rsv

You will need the -r to complete the rebuild of your device trees, the -s option is just for safety and troubleshooting purposes, -v is not necessary but at this point you may want to look at verbose output so you can see if you missed something or have other errors you need ot clean up.

It works as long as the recovered system was installed with full OEM option so all the drivers and low level kernel stuff for all supported systems is there. You will have to keep in mind that you can only port a system from a pci based system to a pci based system or an sbus based system to an sbus based system. The full OEM install may or may not cover everything you need to transfer between pci and sbus hardware, but I wouldn't reccomend trying it as you will probably run into more issues than you'd want to troubleshoot. Otherwise this procedure has proven reliable in most DR situations- we Recovered 5 systems across mixed hardware with this method last night once we properly identified the real issue.




Vincent Esposito

 

[sunny1504]

I guess this could have been achieved by only doing the below mentioned steps

mount with cdrom and mount the root slice on /a
# mount -F ufs /dev/dsk/<root disk device name> /a
# rm -rf /a/dev
# rm -rf /a/devices
# devfsadm -r /a -p /a/etc/path_to_inst