Windows can't load roaming profile....Please HELP!!

[jmajorz24]

I am one of the network administrators for my company. We are currently running a 70 servers citrix farm with metaframe XP 1.0 feature release 2 over windows 2000k boxes. Every day we are getting 50-60 complaints from users who are getting the message unable to load roaming profile you will be logged on using a local profile. The users appear to be random and we are spending much of our day, deleteing user profiles and having them log back in to create a new one to fix these random problems. After the profile is deleted the user is ok and the problem is fixed for that user. However the next day we get 50-60 more of the same problems. Can someone please advise on what could be causing the problem and give me some suggestion? Thanks for all your help!!

 

[BeerGood]

Do you have any NT4 servers and/or workstations? It is possible that the roaming profile is being corrupted which is in turn causing confusion on the win2k terminal servers..... alternatively it could be a rights issue due to a clash between the global network and local server policy... or it can be caused by insufficient registry space.... or it could simply be your domain controller is overloaded..... can you check the error logs on one of the servers and note down exactly what error message(s) are appearing?

Cheers

 

[jmajorz24]

Hi thanks for responding. We are soley a win2k server citrix farm. All of our user profiles are stored on a member server outside our farm. We have thin clients for the client machines. Its a diskless computer operation solely on terminal services. There is no hard drive in the thin clients. I will go over today and go through the error logs on a few of the citrix boxes on the farm and will respond to you with the error logs. Thanks for your time and effort!!!

Jason

 

[jmajorz24]

Hi I have written down the error logs from all of our 70 citrix farm servers. The errors are the same across all 70. We are getting the following event ID's

1.Source: Perflib
Event ID: 1015
The timeout waiting for the performance data collection function
"TermService" in the "C:\WINNT\system32\perfts.dll" Library to finish has
expired.
There may be a problem with this extensible counter or the service it is
collecting data from or the system may have been very busy when this call
was attempted.

2.Event Type: Error
Event Source: Userenv
Event Category: None
Event ID: 1000
Date: 7/29/2003
Time: 12:48:29 PM
User: NT AUTHORITY\SYSTEM
Computer: WS2
Description:
Windows cannot unload your registry class file. If you have a roaming profile, your settings are not replicated. Contact your administrator.

3.Event Type: Error
Event Source: Folder Redirection
Event Category: None Event ID: 101
Date: Date
Time: Time
User: Domain\User
Computer: Computername
Description: Failed to perform redirection of folder Desktop. The new directories for the redirected folder could not be created. The folder is configured to be redirected to \\Computername\Redirected shared folder, the final expanded path was \\Computername\Redirected shared folder. The following error occurred: This security ID may not be assigned as the owner of this object.

ANY HELP WOULD BE GREATLY APPRECIATED.

 

[BeerGood]

1. I assume that you're running the performance monitor on the servers to track usage - that's where the 1015 error is probably coming from. If you're seeing a lot of these error messages then it's possible that the servers are overloaded.

2. I reckon this is the error message that's directly related to your specific issue (although an overly busy server wouldn't help). This is fixed, apparently, in win2k sp3 or by "turning on the IPSec Policy agent" (

http://support.microsoft.com/default.aspx?scid=kb;en-us;319909)

3. Check out

http://support.microsoft.com/default.aspx?scid=kb;en-us;291087

Cheers

 

[DaveRS575]

Did you say ALL of your user profiles are stored on ONE member server (which I assume is a W2K machine) If you have a 70 server Metaframe farm you may want more than one single profile server, (Bottleneck on the NIC there for sure, even with a multi processor multi NIC machine) just my 2 cents though. I assume with a farm of that size you are running XPe and you have thousands of users correct? (If you have 10-15,000 concurrent users, a 50 ticket a day profile issue is probably the least of your concerns!lol )

I assume also that you are using AD and not mixed?

It's tough to tell from these errors, but a roaming profile is an microsoft problem, not a Citrix problem, as you probably know. Roaming profiles and TS profiles are also, as you know, different. Kinda.. My question would be is this a TS profile issue, or a Roaming profile issue?

If you are not using Active Directory, and the profile server is not in AD this could be your problem... If all are in AD, then we can rule out some of the weird stuff that happens in mixed mode. Are all of our metaframe servers running in native mode also?

If all is Kosher in the 2K front, and considering the size of the farm, and assumd usership (that is a very big Citrix farm) I would look into decentralizing your profiles to specific regions, and see if this resolves some of your problems. You could spread them across your many Citrix servers for example...

UNless what you are telling us is that the profiles being stored on the Citrix servers themselves are problematic. Is this a clustered environment as well?

Sorry for all the questions...sometimes my train of thought becomes derailed...

 

[Ogi]

This might help!!

http://support.microsoft.com/support/kb/articles/Q281/2/67.asp

Cheers,
Carl.

 

[jmajorz24]

Yes all of our profiles are one one Win2k member server. However I dont think the NIC is a bottleneck. The reason I say that is because we are using a one gigabit nic. We have ran performance monitor on the sever and network bandwith and usuage is not an issue. The profiles are stored on a member server and we are running strictly native mode. (Win2k across the board). Storeing profiles on individual servers prevents a problem because users dont always log onto the same citrix box. One day they may hit box 34 the next 32 and so on. I have suggested many time to decentralize profiles by zones, but the upper admin say there are many organization bigger than us functioning off one server. It is a dual processor server with a gigabit NIC. The nic itself was 2000.00. We are at the point of opening a case with microsoft, but if I with you guys knowledge and help can find a solution it will sure help me move up the ranks. Futhermore how can I tell if this is a roaming profile or a terminal service profile issue? The errors all point to a roaming profile, however when setting up each individual user, under the terminal service profile tab in AD users and computers we add mapping pointing to the server that store the user profiles and home folders, both of which are stored on the same server for every users. Yes we do have 5,000k plus users which could be a problem but looking a perfomance monitor and monitoring network and processor usuage it doesnt seem to be a problem. If you guys know of any other counters that I should be looking at and the normal statistics of each counter and would like to offer suggestion I would greatly appreciate it. Once again thanks alot for all your time, effort, and shared knowledge.

Jason

 

[jmajorz24]

FYI we are running service pack 4 on all servers the issue was present before service pack 4 when we were running service pack 3 as well. Also I am thinking it could be a GPO thing, reason I say that is because when we just had one GPO at the domain level all was ok or seemed to be, but as we implemented a more robuss GPO problems started. Any ideas on this???

 

[DaveRS575]

Ok....

Here's the problem as I see it...

You have got 5000 users writing roaming profiles, TS profiles, and users Homedrives all to one dual processor, single gigabit Nic box!!?? The 1 GB NIC is nice and fast, and decent pipeline, but its still a bottleneck when 150 or 200 profiles are being written to it simultaneously. Trust me on that one. Your collision domain has got to be fun to watch! lol Perfmon might look good, but i dunno man, that sounds like an awful lot of traffic for error free profile writing unbder the circumstances. Perhaps it looks good because the NIC is only taking what it can? Have your network guys tell you the same story and I'll be convinced that's not an issue. Look at your hard drive activity on PM, I'll bet its crunching away like the wicked witch of the west on crack.

The idea behing a TS profile server is to keep a central copy of your profile for reference. Your userdirs on the metaframe servers, if holding an older profile gets updated each time the user hits "box 34" for example. If they normally hit box 32, that profile was updated last time they logged off. Point the users to the citrix servers for profiles, and point their ts profile to a central box (but at least have two boxes (I don't care if it's your workstation, it will be better than one 2 processor, single nic box)

My farm is spread over the globe, from Tel Aviv to Southern California, to NY, to Singapore,to London.. distributed profile management is a must, but no worries, it's really not that difficult to manage. You may have to build a small admin scrpit to delete or clean up profiles periodically from the global community, but you will have very little in the way of problems (Provided the Default user profile will suffice for you)

Overall... are the same users calling week to week with having this problem over and over or is it random users?

Dave

 

[BeerGood]

Hmmm..... 5k users shouldn't be an issue if they rarely logged on/off but of course what happens is most people will come in & leave work at around the same time so as Dave says.... if it is the problem, then perhaps you could put another nic in the server and team it..... still, the nic might not be an issue but the network itself could be, particularly if some of the profiles are quite large (what would be the largest and average size of the profiles, do you know?) - are all the citrix servers located near the profile server (eg on the same switch) or are some situated elsewhere? The other thing to check would be disk performance on the profile server - perhaps it's struggling to keep up. Monitor %Disk Time and Average Disk Queue Length - if the former goes over 100 or the latter over 1.0 for any great length of time it's time to start getting concerned.

Cheers

 

[DaveRS575]

I second that.. expecially on the Profile size.. Large cumbersome profiles can clog up a network faster than a Wendy's Triple Bacon mushroom burger will your arteries. Give your network guys a call and have them put a sniffer on that segment. they may be able to give you some more specific network metrics and in the meantime, start prepping a spare workstation to be a secondary profile server (You should have a backup anyway) Next 40 people who call, check the average size of their profiles, and push them off to the new server temporarily.. See if they continue to have problems.. My suspician is, their profiles issue (unless they have ungodly sized profiles) will miraculously go away... Continue doing this until the calls stop... You might be pleasantly surprised. You wil then have the ammunition to go into your management office and ask for more $$ to fund the profile server project you came up with, in order to save them money and support costs! Then lay off 50% of your IT people and get nominated for CTO.

(no..I'm not bitter at all)

LOL

 

[jmajorz24]

Hi guys thanks for all your help and support. After doing further reseach I found the gigabit NIC wasnt even configure correctly so we were pushing 80% of our users through the 10/100 nic. I disable the 10/100 nic and reconfigured the gigabit NIC. Futhermore we found a policy in the GPO that wasnt enabled that makes the local profiles delete of the local box. Once the NIC was configured and working correctly and the profile policy enabled to delete local profiles off the box when the user logs off in the GPO and a script written to delete all the current local profiles off the box, the problem is now down to 10-20 users per day with this issue as opposed to 50+. We are thinking the 10-20 per day coming in now is probably profiles that were already corrupted with temp files. We are hoping once we they all filter through that it will eventually come to a halt. I futher agree our profile server houseing user profiles and home folders for 3000+ users is way overloaded. To those who wanted to know profile sizes they dont tend to be to awful big. They avg between 3-15 meg. with the quota set at 30. We are experiencing alot of issues right now and i just have to agree the profile server is maxed to its limits. I have suggested adding another server but all the response I get is how are we gonna divide it up?? Im just a baby admin trying to move up any thoughts suggestions would be greatly appreciated.

FYI other issues involve disconnected sessions not going away, printers not creating when logging on, open files not closing themselves, slow connections, icons not publishing themselves on the client end at random, etc, etc. I think alot of it has to do with the profile server being to large. The profile server and the citrix farm are connected through the same switch to those who want to know that info. All your help and suggestions are welcomed thanks all for all you do!!!

 

[jmajorz24]

to those who wanted to know about perfmon results on the disk I monitored those counters suggested by beergood (%Disk Time and Average Disk Queue Length) there were frequent spikes up to 100 but they didnt stay there long at all. I monitored avg disk read/write times and they were maxed nearly all the time. Hope this will help!!

 

[DaveRS575]

See what I mean? Like the wicked witch of the west on crack.. Glad to hear it's doing better.. Do recommend to your management more than one profile server, even if its just a variety of workstations, it would be far better than one central point of failure or bottleneck. Imagine if all 500+ users called the helpdesk at the same time? :0

Hehe.. I been there .. believe me, you dont' wanna be!

Good luck!

Dave

 

[woodsdarren]

Hi,
Please check out this link as it worked for me with a single Terminal Server.

http://www.asitis.org/archives/000134.html

Hope it helps!