Exchange DataBase dangerously large. Suggestions?

[mlc9]

We are running Exchange 2003 Standard edition with approx 120 mailboxes currently. The database size (priv1.edb) is sitting at 50gb, and the stream (priv1.stm) is almost 12gb. To allow these sizes, I have unfortunately resorted to changing the registery value to allow up to 70gb (75gb is the limit with SP2).

We currently institute no mailbox size limits, though that is my goal very soon. A handful of users have boxes between 1-2gb, and I am working toward helping them archive to reduce. Although I have made great strides in archiving users mail off, and deleting entire boxes of terminated employees, the DB size continues to get incrementally bigger. It has grown almost 2gb in less than two months, and this takes into account all this clean up work I've done.

Everything I am reading points to an offline defrag, but that concerns me. Specifically b/c our business has a 24/7 Ops center that relies on email. The dismount and defrag would take 12-15 hours, and I just don't think we can do without email that long. Plus, I fear I do not have enough hard drive space that an offline defrag will require. Can anybody suggest anything else to guard against this DB growing larger? Why is it growing when I am doing nothing but reducing mailboxes and their sizes? Thanks

 

[mjgolli]

You have 120 mbxs and are 50Gb? Holy cow! I've got 170 at 16Gb....I thought we had problems.

Check that people are emptying their trash and delete any unnecessary sent items.
Check your keep deleted items setting. 7 days is pretty good so deleted items can be gotten back within that time.
Institute a quota. My users get warnings at 500Mb.

There are a couple of options...
1 - Offline the store and defrag
2 - Do a complete backup, make sure it worked, delete your db and restore
3 - The easy option...
Make a new server with Ex2k3Std and add it to your organization. Move mailboxes indiscriminately. Your users won't notice the change if they are not logged in when you move the box...the Outlook config automagically notices. Once you move everyone you can move them back to the originating server the same way. Then check the db and stream size. As always, do a full backup first.

One thing that helped us was teaching our power users to archive with PSTs in Outlook. Another help is to have people save important attachments to their home dir, hard drive or department network share rather than keeping it in the Ex db. It is amazing the savings one gets when lots of AutoCAD dwgs disappear from the Ex server!

As far as space is concerned...delete, delete, delete! and quotas are your friend.

Hope this helps,
Mike G., MCSE MCDBA CCNA

 

[mlc9]

Yea, I realize the absurdity of our number of users vs. DB size. I was just bummed that I had gotten rid of many gigabytes, and yet the Exchange DB still grows. I just worry as it approaches the 75gb limit.

 

[58sniper]

Push out a GPO that empties the Deleted Items folder when the user closes Outlook.

Institute mailbox limits. I read a study that said industry average is 100MB per user. Even if you have some with higher limits, this will get people to start cleaning things out.

Look for event 1221 in your event logs - one for each database (including mailbox and PF). Those will show you how much space you're actually using (the actual size of the two files combined, minus the amount in the 1221 event is the total you're actually using).

Pat Richard
Microsoft Exchange MVP
Contributing author The Complete Reference: Microsoft Exchange Server 2007

 

[mjgolli]

You can set quotas on the organization. They will get a reminder email message at X size, they will not be able to send at a larger X size, and no send or receive at an X size. The sizes are of your choosing. You can have any or all of the options.

My default is a reminder message at 500Mb. Some of my users don't have a quota...these power users use huge graphics or documents and have to keep them together for a project lifetime. Once done, thankfully, these users are smart enough to PST them off the server.

<RANT>
500Mb is waaaayyyy too much space for a normal users email, IMHO. I would, personally, set stricter limits in my org...but the powers-that-be want to be nice for morale purposes.

Abuse of email is not a perk, thank you very much!

Some of my users started complaining that they did not want to get the email that tells them to clean their mailbox. They should not have to do this they would say. One of these people, for example, had 33,000 messages in their inbox dating back to 1996. Woe unto me for the mess I inherited...
</RANT>

Mike G., MCSE MCDBA CCNA

 

[mlc9]

Sounds like the quotas I planned on implementing need to be done sooner, rather than later. I still fear, though, that this will not reduce enough and that I'll need to revisit an offline defrag at some point. That scares me.

While I am making the effort to get people to archive, I run into an issue with laptop users. Some of our users will take their laptops home of an evening and connect back to our internal network via a direct circuit or a VPN client. If they archive their email PST file to their laptop HD, accessing it is not an issue. What you risk, though, is a hard drive failure and loss of PST file.

Like most, they could archive to a shared drive on the work network, but accessing that shared drive from home is sometimes slow. Offline file replication is not an option, b/c it will not replicate a PST file back to the shared network drive. Can anybody suggest a way of archiving a PST file locally, and somehow automatically copying it out to a shared drive on the network? Any other suggestions of reducing the Exchange DB would be apprecitaed as well. Thanks for everything so far. You guys are a big help to the LAN I inherited.

 

[58sniper]

Figure out how much whitespace you have. If it's not over 30% or so, I wouldn't even bother doing an offline defrag. Remember - whitespace isn't a bad necessarily a bad thing.

.pst files are truly evil, and have no place in an Exchange environment. Search this site and you'll see this discussed many times.

Pat Richard
Microsoft Exchange MVP
Contributing author The Complete Reference: Microsoft Exchange Server 2007

 

[mlc9]

If PST are undesireable, then what is the alternative, if any, for archiving? You've got to free up space on an Exchange server somehow, right?

 

[mjgolli]

You could set up a logoff script to copy the PST to a network share when the person logs off. You have to consider, though, that you would be trading one form of network storage for another.

If you have full backups of the ExchDB and do individual mbox backups, there really may be nothing to worry about. Exch is designed to handle up to 75Gb in Standard, and multiple <=75Gb data files in Exch Enterprise.

If your server is suffering slowness or is degrading in some fashion, then more drastic action would be necessary.

For remote users, you should give OWA a whirl. The access is really fast compared to using Outlook remotely and if you use IE, it function almost like full blown Outlook.

What your users really need to do is purge unnecessary stuff. Get important emails into an archive PST, get important attachments saved to a network location and deleted from the db, purge sent items, etc.

Users don't know what is important until they are shown or told. If you don't be a little forceful with them and make them clean up their mboxes they will just collect crap forever. We had to be hard-a$$es with everyone in the entire company...directors to peons...to get our exchange cleaned up.

We ran into the 16Gb limit in Exch 5.5 a while back. Boy, that was a treat and a half! After humiliating people publicly...sending an email to all with the top 20 total item and total size offenders every week...people started to get in line. No one...not even our HR director needs 33,000 emails in their inbox...back to 1996...most of which were crap like "What's for lunch today?" and quickies like "Are we still meeting at 4?", etc.

Of course, you'll have to play it by ear. Depends on the culture of your company. Can you admonish a director and call them an idiot to their face? or do you have to be subtle? Tailor the social-engineering approach to what you are comfortable with and what your superiors will tolerate.

Getting your users in line will probably be the biggest help.

Enjoy,
Mike G. MCSE MCDBA CCNA

 

[58sniper]

Who says you have to free space up on Exchange? Are you running low? Can you add more? What's the retention policy?

There are a lot of alternatives, including increasing storage on Exchange, an enterprise archiving solution, etc.

Pat Richard
Microsoft Exchange MVP
Contributing author The Complete Reference: Microsoft Exchange Server 2007

 

[mlc9]

The biggest reason for archiving and freeing up space is to just help with my original problem of reducing the size of the Exchange DB.

mjgolli (Mike G)......I like your angle on the corporate email environment, something we all struggle with. Everyone feels a sense of entitlement to keep every single email. Remote users really aren't an issue, but rather those in the office with laptops that take them home nightly and work from there as well. OWA is widely used, but that would do nothing to solve the problem of accessing a PST file from a shared drive on the corporate network.

It's just time to get tough,and implement mailbox size limits. I just hope that is enough to reduce the Exchange DB so that I won't have to consider an offline defrag.

 

[Zelandakh]

PST files are bad - don't touch them.

User education is missing from this thread. Don't set policy, get them cutting their mailboxes down. Email isn't a file store.

Check the 1221 to see white space as everyone has said, ensure you've got disk space (otherwise Exchange gets grumpy) and if need be, go to management and get Exchange Enterprise - then just start new stores.

70GB is chicken feed.

When your mailboxes start topping 7GB each, come see me and we'll talk about requirements.

 

[mlc9]

Zelandakh......You made me laugh. If I am reading you correctly, I would guess you have many TB's of hard drive space to play with.

 

[Zelandakh]

A fair few yes!

If you have a small company then get a server with lots of internal disks. Dell will rip your arm off to sell you a server with 6 disks that will give you a good server for small companies (up to 30 users) with 300GB useable. Not the fastest in the world but adequate.

Iron Systems have a DAS starting at $3,500.

My last 14 drive RAID array came in at about $6,000 once I told Dell I was going to HP as they were "better". That's 14 disks each containing 300GB 15,000rpm SATA disks with a fibre uplink.

Disks are cheap, users time is not. A good DR plan, well spec'd servers and arrays and a good implementation means they don't spend their time filing, they spend it earning.

 

[zbnet]

I told Dell I was going to HP as they were "better"

 

[mofusjtf]

I'm not 100% sure but I think Exchange 2003 STandard can have up to 4 storage groups with one database store per storage group. Each store has a limit of 75 GB. Why not create another storage group / database store and move mailboxes/users to this new location? You will essentially split the amount of storage space between the databases. Each store will be around 30-35 GB which will ease your mind about reaching your store size limit.

 

[mlc9]

mofusjtf......That is a great idea and would be awesome if true. I am going to research that ASAP! Thanks for all of these posts guys!

 

[ntinlin]

mojusjtf - Incorrect, I believe you are thinking of XC2007 Standard.

XC2003 SP2 standard can have only one mailbox store with a maximum size (configurable via registry) of 75Gb. As mlc9 said in his original post.

mlc9 - If you have sufficient local storage you could always upgrade to XC2003 Enterprise, assuming you can find someone to sell it to you. Not sure if it can be purchased nowadays. It just installs over the top.
Then you can create all the mailbox stores you want. Well, 20 anyway.

Other option is something like Symantec Enterprise Vault which integrates quite well with Outlook and has the option of 'Offline Vaulting' to replace PST's.

But as it says it is an Enterprise solution and would be a big project in its own right, not a quick drop-in.

Neill

 

[dangerousdom]

We use Quest Archive manager, its a similiar product to Enterprise Vault

 

[mlc9]

All good thoughts, but my company does things on the cheap. I would love a third party email archiving solution, though.