Citrix farm constantly banging on domain controllers

[wdoellefeld]

Hi,

Any insights would be appreciated.

I have a Citrix (with powerfuse) farm consisting of 13 servers all running 2003 (no SP1). I have 2 DCs (both 2003 + SP1, only one GC). For quite awhile now we have noticed high CPU usage on the DCs, lsass being the process of course. As advised by MS we applied SP1 on the DCs to curb the high CPU. According to MS all 3 hotfixes for lsass issues are included in the SP.

SP1 didn't help any.

Doing packet captures we are seeing our citrix servers banging the heck out of both DC's. Approx 60% go to one DC the rest to the other. This is keeping the lsass process running the CPU at a constant 85-90%. We are trying to understand what Citrix is doing and can't figure much out. At any given time we have 4-5 of our Citrix servers actually in live pool but we are seeing every server in the farm hitting the DCs.

Only been with this company for about 3 months now so not sure what has been going on previous. Anyone seen this before?

FRCP

http://www.frcp.net

 

[benace]

What is the nature of the packets? Protocol, ports, payload. What process owns the open socket that's being used to send the packets to the DC? Is the DC returning any
information to the citrix server? how many users on each citrix box? is there constant logon/logoff activity?

If you can identify the processes on the citrix box, use regmon and filemon to see what those processes are looking for.

I know there are several viruses, trojans, and spyware packages that exploit lsass. worth running a few scans of your boxes.

 

[ascotta]

Here it is

Using the Aspnet_regiis.exe utility included with the .NET framework, configure IIS to use ASP.NET Version 1.1.

Go to a command prompt and change to the directory c:\windows\microsoft.net\framwork\v1.1.x

From the command prompt run the following command to unregister ASP.NET.

Aspnet_regiis -ua

From a command prompt run the following command to re-register ASP.NET.

Aspnet_regiis –i

To confirm what version has been installed run the command

Aspnet_regiis –lv





[blue] Oh you know, just doing what I do.[/blue]

Cheers
Scott

 

[ascotta]

Ignore above wrong thread

[blue] Oh you know, just doing what I do.[/blue]

Cheers
Scott

 

[wdoellefeld]

benace.

I'm thinking it's legit albeit a maybe "runaway" process of some kind, a setting we've missed? Not seeing any spyware or attacks in the traditional sense. The servers are clean and secure just busy talking to our DCs for no apparent reason. I'm trying ID what app, service, whatever is using lsass this much. I think the problem is not so much lsass but what is using lsass. 90% of the packets are refering to NETLOGON. The thing is.. all my citrix servers (and only my citrix servers) are doing this and over half of them are idle but I see the same level of packets coming from them as the live boxes.

Looking at TCPView from one my idle Citrix boxes...

I see Lsass with one established connection and 2-3 more connections opening at random every 7-9 seconds to the DC.
Also see (System Process):0 with one steady established connection and 1-2 opening every 20 seconds or so to the DC.

Looking at the processes in task mgr I see lsass.exe running at a steady 5% and islogoff.exe running at steady 3%



FRCP

http://www.frcp.net