Poor RAID1 array performance

[DXBLouie]

I've been noticing lately that my newly setup system is EXTREMELY slow to respond at times!
for instance if i create a new document or photoshop file.. the moment i click anywhere on the screen the software wont respond.. and the windows performance monitor actually shows the process is waiting on disk operation (writing the temp file)

My system is based on the following:
MB: Gigabyte GA-965P-DQ6(rev. 1.0) with Intel ICH8 RAID Controller, and Gigabyte SATA2 RAID controller
CPU: Intel Core 2 Duo E6600 @ 2.4GHz
RAM: 6GB DDR2 800
OS: Windows 7 Ultimate 64 bit

Storage Setup:
Drive C: RAID 1 (Mirroring)
2x Western Digital 250GB SATA HDD (WD2500JS-00NCB1)
(The drives are physically connected to port 1 and 2 of the INTEL ICH8 chipset)

Drive D: Standalone SATA
1x Samsung 500GB SATA (HD502IJ 1AA01110), connected to port 1 of the Gigabyte storage controller

Drive E: RAID 1 (Mirroring)
2x Western Digital 1TB SATA HDD (WD10EADS-00P8B0)
(The drives are physically connected to port 3 and 4 of the INTEL ICH8 chipset)

i thought the additional array is adding an overhead, so i booted up with it disconnected (i physically unplugged both 1TB drives, and the volume E disappeared), i also unplugged the standalone drive D from the other controller, bascially leaving the main volume alone.
No change.. performance was still misrable.

i have enabled read/write cache enabled on all volumes, same result.

i ran a benchmark on the arrays (HD Tach) and the results seem pretty okay somehow:

C:, 250GB RAID1 Array:
Random Access: 13.6ms
CPU utilization: 4%
Average Read: 55.6MB/s
Burst Speed: 1908.1 MB/s

D: 512GB Standalone SATA Drive:
Random Access: 14.7ms
CPU utilization: 4%
Average Read: 77.3MB/s
Burst Speed: 166.4 MB/s

E:, 1TB RAID1 Array:
Random Access: 13.7ms
CPU utilization: 3%
Average Read: 83.8MB/s
Burst Speed: 2089.8 MB/s

Actually the reason i decided to post this, is today i had to copy a few files around 4GB in size to another machine
i plugged a memory stick, and the process took over 55 minute to complete!!!
i had the same misrable performance trying to copy the files over the network or even burn them on a DVD.
basically all read and write operations are extremely slow

(once done, the files took 4-5 minutes to be copied from the stick to the other machine running linux and a similar RAID1 setup)

Any suggestions?

Thanks in advance!

Louie

 

[kjv1611]

Your system RAID looks like it has very slow speeds for modern hard drives. How old are the drives? If you could live with no RAID for the system drive, and just create a restore image instead, then you could run the system from the Samsung 500GB drive (which model is it?), and have a much faster system, regardless. Especially if it's a Samsung F3 drive - VERY nice drives. Though, the system would be a little more responsive with a Raptor or Velociraptor for system duties, and even moreso with an SSD, but here we're talking money.

1. Check to see if a driver update is available for your Intel chipset. There's a tool you can download to check it on Intel's website.

2. Try running a general clean-up tool or two or 3, and see if they fix any issues:
From

http://www.download.com

- Glary Utilities, CCleaner, Advanced System Care.

3. Make sure there's no malware running in your system. With that in mind, also make sure you have an up to date antivirus. If you got a new machine, and it came with a trial of an AV product that ran out, you need to renew, or go with another option. I personally would go with one of the free options: Microsoft Security Essentials, Avira Antivir, AVG, Avast! But for the quickest check on things, download Malwarebytes Antimalware from

http://www.download.com,

and run a quick scan with it, remove anything it finds, reboot if necessary. If it does find anything, you might want to mention what it found here... and if it does find anything, rescan after the reboot. Also, it'd be good to download and do a quick scan with SuperAntiSpyware FREE Edition, also from

http://www.download.com

 

[kjv1611]

Another thing you could look at: resetting the prefetch data.

http://www.coolbuster.net/2009/08/empty-delete-prefetch-windows-7.html

http://lifehacker.com/201453/tweak-windows-prefetch-for-a-faster-startup

And if all that fails, it might not hurt to run a memory test with MemTest

 

[kjv1611]

Oh, and about what I said about the hard drive performance. It's not TERRIBLY slow, but that sort of makes me think that the drives may have a little age on them. For instance, they might be 3 or 4 years old. I've got a handful of WD500AAKS or something like that drives which run very well in RAID setups, but they've got a little age on them. Their read numbers look a little like that if I remember correctly. Most of the latest drives, if not all, will be above that. Some of them are above 100MB/sec. If your Samsung drive is reading at 77MB/sec, then perhaps it's an F1 or F2 at best, rather than an F3 - also having some age on it...

As far as what I mean by age, if it's a new system, then it could still be older technology drives, as often manufacturers will use what they have in stock. If it's a custom build, then I'm guessing you used some old hard drives, and maybe some new hard drives in the same system.

Also, some hard drives are not really built for RAID handling. Some that aren't built for it still do well, but some do awful in RAID setups.

 

[DXBLouie]

Following are the drive specs:
Drive C (250GB) RAID1 components:
WD2500JS: SATA 3 Gb/s interface, 7200RPM, 8MB Cache, 8.9ms seek time

Drive E (1TB) RAID1 components:
WD10EADS: SATA 3 Gb/s interface, 7200RPM, 32MB Cache, 8.9ms seek time

so sure they're not state of the art, but surely they should be good enough to open a 3MB PSD file without the system hanging for 20 seconds!

i've had the same PC running a RAID5 setup in the past with 3 hard drives (3x 250GB) and have had absolutely no performance issues.

i'm tempted to buy me 3 1TB drives to build a new RAID5 array, but again, it would suck to do that only to have the same performance due to some unaddressed issue

p.s: the samsung drive is HD502IJ F1 Series (500GB, 16MB Cache, 7200 RPM)

moving on,
i have Microsoft Security Essentials and Spybot S&D since day one.
i've cleaned up the obvious, and i have no extra processes running that i don't need.

will try the prefetch clean up bit, and will also try to temporarily disable the AV and Malware protection to see if that's contributing to the situation.

Thanks!

 

[kjv1611]

Me again. Just thought about this - on the prefetch data... I looked a little more at the directions on the 2 links I listed above. The one at LifeHacker is older and is based on Windows XP, also it didn't seem to mention an early warning.

Here's the warning from me, and from the other link: don't just go happily deleting everything you find in the folder.

My suggestion here, if you're not sure about it, or concerned, give Tweak7 a go:

http://www.totalidea.com/product.php?Product=Tweak-7

I've not tried that one yet, but I've used TweakVi a lot on Windows Vista systems, and it has worked very well. I wouldn't go checking everything available in the program, but it does specifically have built-in options for handling things such as Windows prefetch data.

 

[DXBLouie]

too late
i already went in and deleted all .pf files.. should be alright

 

[kjv1611]

[LOL]

Most likely, you'll be fine, true.

And yes, your hard drives alone wouldn't explain it.

Since the change happened later, then it sounds like something is giving out or else something changed in software/settings.

On the hard drives, another thing to check into is with your onboard controller, you could be overdoing it, but it sounds like you narrowed that out already, and I wouldn't suspect it with your chipset on a Gigabyte motherboard.

Another thing - did you make any hardware changes prior to this? If you added anything, you could be putting too much draw on your power supply. That would also be worth checking into. But being you removed all but the system drives to test, It's also likely not the issue.

It might also be good to try running a single drive of your RAID1 system array as a single (if it'll run)... particularly changing the BIOS method from RAID to AHCI... if Windows will load, and run, you can test to see if it runs better as a single. If so, then you can narrow it down to being that hard drive, or else being a RAID issue, which it honestly doesn't sound like at this point.

Given it happened at a certain time, and you've narrowed down some hardware possibilities, I'm thinking it's software.

Another test for that is to start in Safe Mode, see if same problems exist, try to send files to or from the USB drive as a test. If the problem vanishes in safe mode, it's either a virus/malware, or it's some other system setting.

Another thing to check is back-up software as well as any system monitoring software. Also, see if you can check for any failed Windows updates.

On Spybot, it aint the best tool anymore. Tests have shown it doesn't pick up as many of the baddies as it once did. Frankly, the only app I use from a few years ago, currently, on a regular basis is SpywareBlaster. I used to recommend: Ad-Aware, Spybot, and SpywareBlaster, but the first 2 got too weak, and oftentimes caused performance issues... there cons outweigh their pros. However, I could see using them as a back-up scanner if nothing else.

For a quick scan, download, install, and run a quick scan with Malwarebytes Antimalware from

http://www.download.com

- if there's anything much in your system, it should get picked up, at least partially if not fully, by that program.

Then we can go from there.

 

[wahnula]

What I don't like is that you apparently are using the "Matrix" function of the ICH8 to control (2) independent RAID 1 arrays. Onboard RAID controllers are not true hardware RAID, they use the CPU to process RAID, so that's a lot of cycles between the two arrays.

I would remove one of the arrays from the ICH8 and see if performance improves. Try copying a large file from the C: drive to the D: drive. If that works, you've found the culprit.


Tony

Users helping Users...

 

[DXBLouie]

kjv1611: to be honest i think i've had the issues right from the start, but always thought it was photoshop's fault, till one day i got fed up of it and tried to get to the bottom of it, and noticed storage being the bottleneck..

i ran all kinds of tests since my last message.. no malware/spyware whatsoever.

i wouldn't mind removing all the other disks and turning the BIOS mode from RAID to AHCI.. but i'm scared shitless that may mess something up when i'm done with the test and i'm to put the array back together :S

wahnula: agreed that the software controller puts strain on the CPU, but that's not the case here..
i've had the same controller running two arrays in the past (RAID5 + RAID1) and all was well.
the only change since then is that i took two out of 3 discs from my ex-RAID5 array, and configured them into a RAID1, bought 2 new 1TB drive and setup a new RAID1 array with those too.

anyway, i did disconnect all the additional drives keeping one array with a total of 2 drvies. and that made absolutely no difference.

 

[wahnula]

What does the task manager look like? Any processes pegged?

Tony

Users helping Users...

 

[DXBLouie]

i ran some tests copying a 1.5GB file across from one drive to another back and forth and following are the results:

Legend:
C: (2x 250GB RAID1 Array)
D: (1x 500GB SATA Drive, standalone)
E: (2x 1TB RAID1 Array)

E -> E ~125MB/s
E -> D ~70MB/s
E -> C ~2MB/s

D -> E ~125MB/s
D -> D ~60MB/s
D -> C ~2MB/s

C -> E ~15MB/s
C -> D ~20MB/s
C -> C ~2MB/s~

Something is seriously wrong with the C array..
now i'm thinking perhaps one of the drives is dying and the ICH8 is unable to tell?!
are there any S.M.A.R.T tools that can read the status of a physical drive behind a RAID array?

 

[wahnula]

As for S.M.A.R.T data, that depends on your RAID driver, I'm not sure about yours, but I would look in the Event Viewer for clues. I had a problem with a SATA controller reverting from IDE to PIO mode, so check that in Device Manager. Change the cables on both drives just to be sure that's not it.

A quick test of the individual drives would be to boot to one only, then the other only, and see if there's any speed difference. If so, leave the better one in and rebuild it to a new drive. If they're both the same then it's a controller/driver/other hardware issue, and wait until bedtime to rebuild!

Tony

Users helping Users...

 

[kjv1611]

See, now another recommendation to test one drive separately.

At this point, what have you got to lose? Give it a go. Your data is safely on another drive/array anyway, so if it did go crazy in all your testing, you would be fine. Of course, he's recommending just leaving it at RAID mode with one drive, so that would keep the drivers consistent, and it would not remove the RAID controller as a possible culprit - good to test that way at least initially.

If you have the money to burn, buying one more small drive, or 2 small drives that are new will be faster anyway. But it'd be good to verify whether it's the drive(s) causing the issues or the controller or something else.

But since you tested out fine on hdtach - not great, but not as bad as your last file transfer test, I'd be more inclined to think it's some software issue.

Did you ever try safe mode booting to see if you get the same issues on file transfers while in safe mode?

 

[DXBLouie]

So i plugged the drives into my linux box and ran smartctl.
one of the discs reported a few errors, namely: REALLOCATE_SECTOR_ERROR and SEEK_ERROR_RATE being higher than the threshold.

kinda strange the intel matrix controller didn't pick up any warning though.

anyway, i bought two new drives, nothing fancy.. just what i found right away: WDC WD5000AAKS-00v1A0 (500GB, 7200RPM, 16MB Cache) setup a new array and reinstalled windows.

happy to say i'm getting at least 30MB transfer rate between the two arrays on the same controller.

now i'm certainly looking for a proper hardware controller to replace the ICH8R.

however my motherboard (Gigabyte GA-965P-DQ6 (rev. 1.0)) also has an additional gigabyte RAID controller which i've never enabled (i use it in standard IDE mode as i never bothered looking for its driver)
then again i'm not sure if i can transport an array over without having to recreate it and destroy my data..

 

[wahnula]

now i'm certainly looking for a proper hardware controller to replace the ICH8R.

Excellent decision! I highly recommend 3Ware.

however my motherboard (Gigabyte GA-965P-DQ6 (rev. 1.0)) also has an additional gigabyte RAID controller which i've never enabled

Leave it be. As I've learned the hard way onboard controllers are not as good as true hardware controllers. Besides, with a true hardware controller the RAIDing will be offloaded from the system CPU. A good hardware controller like the 3Ware Escalade series can handle both arrays. Expensive, but US-based support and superb reliability come at a price.

As for re-using the existing array, forget about it. Build the new array with the new hardware controller and start fresh. You might be able to ping-pong clone it over.

Since the drives are so large, I would recommend partitioning a 50 GB or so partition for OS & apps, and the rest for data. This way you can reload or change the OS without affecting the data partition.

Tony

Users helping Users...

 

[DXBLouie]

Thanks Tony,

i actually did partition the new drive setting aside 200GB for the OS and apps. and 300GB for my documents and other files.

i did try to enable the Gigabyte SATA RAID controller.. and while it has a nice BIOS interface.. it couldn't migrate my Array or create a new Array with the old discs.. meaning i'd lose my data so that went out the window.

thanks for the hardware controller recommendation.. i'll certainly check out 3Ware, although it would certainly help if i can migrate my arrays over

 

[DXBLouie]

i spent a few days searching for good controllers and made my mind on an Adaptec 4805SAS PCIe controller with 128MB DDR2 cache, supporting up to 8 SAS, SATA, SATA2 drives (or 100 with expanders)

only pain in the butt is having to rebuild it all from scratch, but i must say i'm quite excited!

 

[kjv1611]

but i must say i'm quite excited!

Great starting place! So now it'll take you a good 2 weeks to get to the point to where you pick up the computer, and throw it as far as you can, then go get the 12-gauge.

 

[wahnula]

Good for you! I would build with SAS if I had the option. I know you've already bought SATA HDDs but next go round try SAS. More expensive, but built like tanks with insane MTBF numbers.

Moving data arrays is pretty easy, they will recover from clone or backup quite well. Just use a spare HDD to clone the data to, test the backup, then blow up the old array & build the new. Recover from clone or backup and you're off. Theoretically, this should also work with the boot array, but as you've just built it I would start from scratch with that array.

Tony

Users helping Users...