RAID array of drives completely wiped 3

[BiggerBrother]

last week my server suffered catastophic data loss. the basic server setup is one server with RAID with four drives attached.

the sequence of events was a series of input/output read/write errors where the drives had reportedly gone into read-only mode. the server was rebooted on three occassions over the week which temporarily fixed the problem on each occassion. each time, all data was recovered

after the fourth and final read/write error, my host advised me that an FSCK should be run. my host ran an FSCK - during this process i was advised that a new RAID controller would be required. Within 15 minutes my host advised me that all hard drives attached to the RAID controller were completely blank (0kb). no operating system was present.

i would be grateful of any advice on what could have caused this catastrophic failure.

BB

 

[LawnBoy]

An idiot tech and no backup tapes?
Depending on your SLA, you may be able to sue for breach of contract...


"We must fall back upon the old axiom that when all other contingencies fail, whatever remains, however improbable, must be the truth." - Sherlock Holmes

 

[gbaughma]

Sounds like a hardware failure... possibly a failed memory chip on the RAID controller. Could have been caused from a power surge, heat, any number of things.

Raid 5 is usually pretty robust in handling a failed drive; but a failed controller is another issue.



Just my 2¢
-Cole's Law: Shredded cabbage

--Greg

http://parallel.tzo.com

 

[LawnBoy]

I don't know of any RAID array that will survive being placed on a different controller. The new controller is immediately going to initialize the disks.

Sounds to me like the tech just plain f*cked up.


"We must fall back upon the old axiom that when all other contingencies fail, whatever remains, however improbable, must be the truth." - Sherlock Holmes

 

[macd68]

I was a data center tech at a large hosting facility for a while and had plenty of experiences with failed raid.
Off the top of my head:

1) A failed controller doesn't wipe disks.
2) Even if the controller failed an identical controller installed will allow you to recreate the array without touching content on disk. Some RAID controller bios (old PERC's) have no internal functionality for this however.
3) If a failed storage chain component was writing garbage to disk (possible) then you may have corrupted data. This is not the same as a wiped disk.
4) No os present is not the same as no data on disk. The MBR may have been overwritten in this case. The simplest explanation is the array failed and someone hadn't rebuilt it correctly.

My take is this: A tech rebuilt the array configuration on a new controller and then initialized the disks by accident. This is not at all uncommon if you're tired and have six things going on at once (my experience as a midnight shift guy says this is pretty routine). I never did it, but I did replace a bad disk with another bad disk on two occasions resulting in a chain of failures. The customer was not happy. :/

 

[BiggerBrother]

And thats the best thing about this site - completely un-biased opinions! You have no idea how much these responses have helped!

Stars all round.

BB