RAID catastrophe (hypothetical) 1

[joed2k]

Sorry for the length here, but I know you need details to help...
I'm somewhat new to the whole IBM Server thang, but I'm jumping in with both feet and have come up with a possible problem, but no solution yet. Maybe I'm missing something, but I haven't found a way around this scenario without losing data, which is exactly what we don't want to do.
Please read this over and comment, if possible.

Let's say I'm using a ServeRAID-4H (2 internal, 4 external). I'm using channel 2 internal to control the box, which leaves channel 2 external unavailable. Let's also say I'm then using channel 1 external to control half of a EXP300 and channel 3 external to control the other half of the EXP300. That obviously leaves channel 4 external available.
There are 6 HDD's in the EXP300, 3 connected to the 4H channel 1 and 3 to channel 3. All 6 drives are part of a single 5E array, one logical drive. Data is stored on this logical drive. OS is Win2K, ServeRAID 6.11.

Therefore, the array will show 1 logical drive with 3 physical drives on channel 1 and 3 physical drives on channel 3.
If channel 1 of the 4H was to take a dump (theoretically), the 3 drives on that channel would become defunct, correct?
Also, both the logical drive and the array would be inaccessible.
Since part of the definition of the array is 3 physical drives from channel 1, and channel 1 is dead in the water, how would the data be recovered?

Is there a way to switch over to the available, functioning channel 4 and change the array definition, and still be able to recover the data?
I purposely caused a scenario similar to this, and was unable to recover (experimental, so not a problem).
What worries me is, why couldn't I recover?
I'm wondering what I'm missing?
It seems to me that there must be some way to recover the data in a scenario such as this, but it seems that the only way to rearrange the configuration was to wipe it out first, and recreate the array. I'm quite sure the data was still on the drives, but I couldn't access it.
Any ideas on where to look for info on something like this?

Thanks for taking the time to read my blathering...

 

[ibmtech65]

If you were to lose channel 1 you could attached the cable to channel 4 and then when the server boots up you will get a message that the drive locations have changed and do you want to accept the new locations you would accept the changes and should be able to access that array again.
Just a word about a raid 5E, it incorporates the hot spare across all the drives so if you were to lose 1 drive the array is going to have to do a compression of the data before the rebuild will start and then it will have to uncompress the data back across the array once the drive is replaced, depending on the size of your drives this can take several days. Versus a hotspare drive where the rebuild will only take a couple of hours depending on the size of the drive.
Since a normal hotspare can not be assigned to any specific array I would suggest you create normal raid 5 with 5 of the drives and keep the 6th has a hotspare, that way if any of the drives in any of the arrays goes defunct you have a hot spare it can rebuild on while you wait on the replacement drive. Odds that you will lose a drive are greater that you will lose a single channel on the raid card.
And I would not split the back plane on the exp300 since you are using it for a single server. If you were to loose a channel you could still move the cable to another channel and get access to the array same as before and you would be increasing your performance.

 

[joed2k]

Thanks for the reply.

Actually, I do have a hot spare in the box.
Array A is raid 1 for the OS, array 2 is 5E, 6 drives.

I was under the impression that the 6th drive in array B would be for exclusive spare use within that same array, but experimentation has shown me that, if there is a hot spare available, disabling a drive in array B results in the array defaulting to the avaiable hot spare, rather than using the built-in.
I've since learned that this is by design, but I didn't expect it.
Under those circumstances, would you still recommend 5 over 5E?
Is it possible to then force the data back to the previous location, once the failed drive has been replaced?
Also, I thought I would have better throughput to the EXP300 by using 2 SCSI channels in parallel rather the a single channel as you suggest. Am I wrong with that assumption?

Thanks.

 

[ibmtech65]

You are not going to get any faster through put by using more then one channel because the data still has to go through the single bus on the system board.
A hotspare does not get an Array assignment, it is sitting there for use by any of drives that are on that controller card. If you lose a drive in the Raid 1 the hotspare will rebuild for that also. The only way to keep a hotspare in the same location once a rebuild has been completed would be to pull that drive you want to remain a hotspare which would force a rebuild to the current hotspare, once the rebuild is complete you can put the drive back in and it will become the new hotspare.