Store.exe consuming large amounts of RAM

[ShawnF]

Hello,

W2k server running Exchange 2000. Store.exe has been consuming up to 1GB of RAM (out of 2GB total), and appears to be slowing the server down. I don't recall Store.exe ever consuming that much RAM before and it seems to have started this past 7 days or so. If I stop and restart the service or reboot the server, it goes back down to about 70MB usage and 1.2GB free RAM. It can run all day like this, then when I come in the next morning the memory usage spikes again. Is store.exe supposed to consume this much RAM?

I've done searches online and read things ranging from it's normal for store.exe to consume as much RAM as possible (on Exchange 5.5 though), to problems occurring when trying to run tape backups during the default time of mailbox store maintenance of 1am-5am. I'm not real sure what to believe becuase as I said this seems to have just started this past week. I ran a full virus scan on the server and didn't find anything. Microsoft has articles on CPU consumption being up to 100% with store.exe, but that was with Exchange 5.5 again and our actual CPU usage is normally under 10.

Thanks for any help!

 

[fs483]

I'm actually noticing the same thing on one of my exchange servers. I too have 2 gigs of RAM and yesterday Store.exe took up to 1.7 GB of RAM. This morning I rebooted the server and the RAM droped to 50 or so megs and has gradually increased since then. It's causing system lockups for my users that are accessing the server from terminal server. I have about 8 or 9 users always connected and each user always has word and outlook xp opened. The only way to solve the memory usage is to reboot which I'm going to start doing every morning. My server has Windows 2000 server with exch 2000 and all the services packs applied. Hardware is Dual Zeon 1.8GHz with 2 gigs of ram... Do you have a similar config ?

 

[dennisbbb]

It's true that on 5.5 store.exe will use all the ram you throw at it. All our 200+ users are connected 9 hours a day and remote users are connected every 5 minutes via pop/smtp. No problem.

Check if your server is being used for relaying spam. Check on your incoming and outgoing queues with Exchange performance monitor.

 

[fs483]

In my case, no my server is not relaying (checked and rechecked) and the queues are clean. I also have both Mail antiviral software (sybari) and file antiviral software (Norman) running. On thing is that this server holds years of emails with attachment and the users tend to expand the whole tree of subfolders. Their email accounts were getting so big, we couldn't do exmerges anymore and had to split up the archived mail. I think the priv.edb and public.edb are getting close to the limit.

 

[billieT]

Store.exe has taken up as much RAM as it can get since Exch5.5 (something that hasn't changed). It's like SQL in that it will gobble what it can get (without "starving" anything else. Here's a blog entry from the Exchange team that explains it nicely:

http://blogs.msdn.com/exchange/archive/2004/08/02/206012.aspx

(with a pointer to a Support article that talks about monitoring processes)

 

[billieT]

Oh, by the way, where is your paging file? If it's on the same physical disk as the (over large) Exch DBs, that might have an impact on performance

 

[ShawnF]

Akwong,

Our server has similar specs, though it only has one 1GHz processor.

I "may" have stumbled upon the problem. One of our users *sigh* sent out a 125MB attachment.... I didn't know about it right away. In fact, I didn't know about it for a few days until the user informed me they haven't been getting any E-mail. When I accessed their mail using remote desktop connection through a terminal server, I saw the returned message from the recipient's admin account saying the following message could not be delivered, and the attachment was included. The server locked up just trying to read the E-mail and I had to reboot. I deleted the offending E-mail and things were smooth for a day. Then the problem came back yesterday! Then the user again told me they aren't getting E-mails. The admin account from the receipient's E-mail server sent the message back again saying it was non-deliverable. I'm thinking the message screwed up their server as well.

I should point out that the user's mail set up is set up for Internet Mail only, though I've set up two different profiles in their Outlook2k. One for at work, and one for not at work. They download their E-mails off our server, rather than connecting directly to Exchange with Corporate or Workgroup settings. So, despite being "at work" in the facility, they still download internet mail, and the E-mail was choking the system and not letting the user get any E-mails. I was able to log directly on to a terminal server and check the user's E-mails that hadn't been downloaded to the laptop yet and deleted the offending E-mail.

Having found this second returned E-mail, I tried to delete it but the user's computer locked up. I attempted to stop and restart the store but I got a window saying it wasn't responding and it just froze. So, I had to reboot again. Immediately after reboot I quickly went into the user's account and deleted the offending E-mail, then I set up filters on our mail forwarder to block all incoming E-mails from that admin account. I will let it go a few days and see how things are.

Right now after reboot again, store.exe is comfortably sitting at 50MB and 80 threads. We've had these servers for 3 years and I don't ever recall store.exe getting up to 1GB of RAM until this week.

 

[ShawnF]

Dennisbbb,

Dumb question, but where is the Exchange Performance Monitor? I'm sure it's in the system manager, but I don't know where. Our server shouldn't be relaying, as I specifically asked the firm that set it up to make sure it doesn't. However, I ought to check anyway just to be safe.

 

[fs483]

Well we did a test and didn't reboot the server. Exchange passed from 60 MB after reboot to 960MB 24hrs later. Memory usage has reached 1.7 gigs out of 2 gigs... So far no sessions have frozen but I'm not taking any chances and will reboot during the night. In my case the users ARE NOT sending huge emails. They just have many thousands and thousands of emails stored... Still working on a solution, if there is one !

 

[NortonES2]

Exchange will use as much RAM as it can (designed that way), after all you didn't buy that 2Gb to have it sitting there unused.

-------------------------------

If it doesn't leak oil it must be empty!!

 

[dennisbbb]

The solution from the start for us here is to get their junk off to their pc with Personal Folder pst file. They store up to 2 gig, we create them another. These users are so lazy they don't even bother deleting spam, much less legitimate email. It's a "cover your ass" mentality plus "too busy to do that" mentality. They care less about "your server". So our mentality should be "hey you store your sh**", and when it crashed, we say "you jammed up your email by not deleting junk, we gotta create a new one so you can get back on your feet". PST is the only way to go. If you want to protect them from having a harddisk crash, get them mirrored with two drives. Drives are really cheap now.

 

[billieT]

I don't agree with you there, dennisbbb, on PSTs being the solution to your woes with expanding storage sizes. What happens when the stupid users save business-critical info to a local PST, which is then totally lost when the HDD dies and you have no backup? And then there's supporting issues with the damn things, which in my experience is at least twice as many issues when supporting just Exchange+outlook together.

The answer to the ever-expanding storage is to impose quotas. THEN they soon realise that they don't really need to hang onto those idiotic AVIs that their mates send them. If they need to hang onto large attachments for business purposes, provide network storage areas and provide instruction on how to save attachments. If they want an integrated solution, there are tons of 3rd-party apps that will manage docs and attachments, but MS's own Sharepoint does a reasonable job:

http://www.microsoft.com/technet/prodtechnol/sppt/reskit/c4161881x.mspx

 

[dennisbbb]

I agree with you billieT on the business-critical info can be lost to local HD crash for critical users.

On the other hand, these same critical users are too lazy and too busy to manage their own email that they decided to keep everything as "cover your ass" technique. There is no way to keep up with them if things are stored on the server. From the logs and from a catch-all machine (which serves as a critical info backup), I estimated the server reaches 1 gig per month. Even if we have that kind of space, we cannot possibly do any offline backup with the ever increasing backup time required, and eventually the server will run out of space. It's just not a "long term" solution.

 

[fs483]

My large store.exe appetite seems to have stabilized at 9xx megs of RAM and my sessions haven't crashed. The only thing I did was reinstalled Outlook.

 

[billieT]

Quotas, dennisbbb, quotas!!!

and akwong, PLEASE tell me you're not running the Outlook client on the Exchange box!! bad, bad, bad, bad.... :-(

 

[fs483]

Yeah I know bad bad bad but no choice ! This client only has one server. Most of my clients have a dedicated server that takes care of remote users and exch on a different one. However it's still a pretty strong server other than having Store.exe take up what it wants. I haven't rebooted the server in a few days now and everything seems fine except when I run exmerge, the cpu usage drops and memory utilization increases.

 

[ShawnF]

As for my original post, the removal of the huge 125MB attachment E-mail has resolved the lockup issues, though it hasn't stopped Store.exe from reaching up to 950MB. However, it's been running for a week like this at 950MB and hasn't had any issues and performance is quite acceptable. And I keep hering that store.exe is supposed to consume as much RAM as it can as it was designed that way. So for now I'm going to let it go.