"Rogue Connections"

[jdemmi]

We have hundreds of Citrix servers (I am not exaggerating) and we keep noticing rogue connections to them. By rogue connection I mean a user that appears in SERVER MANAGER (on the Citrix server in question). This user is "connected" to resource IPC$ and FROM another Citrix server. They have no open files and if we disconnect them they will randomly be reconnected.

We have all flavors and combinations....
NT 4.0 TS and MF 1.8
2000 and MF XP
etc, etc.

And this happens on all of them....any ideas?

 

[tbhatt]

You should run a program from Systernals that will allow you to see handles within a process. Alot of times a handle which runs below a process as a thread gets hung and cause problems such as yours. This might be the case. If it is, then i would kill the handle and then logoff the user, this should work if it is the issue i described above...

 

[jdemmi]

Thanks, I will try this but I don't think it's going to help. I don't think it will show anything for these "connections". Remember these connections are "coming" from another Citrix server and there is no way possible, in our environment for this to happen.

 

[jdemmi]

As expected, I ran Process Explorer (

http://www.sysinternals.com)

and the only "OWNERS" appear in this utility are those who actually have a true connection (logeed into the Published App\Server). Any more ideas

 

[kgouldsk]

My approach would be:
1. On the server you are seeing the problem, run a "netstat -a" to find out the name of the server the connections are coming from. See if they're coming from a bunch of different ones, or just one.

2. Go to the server the connections are coming from and use Process Explorer to look for anything funny.

3. One other thing you may do is look at the shortcuts on that server. There is a registry setting which controls how shortcuts are created, and if you cloned servers, the shortcuts may in fact all point back via UNC to the original server cloned from. This could create the problem you speak of.

 

[jdemmi]

I can already see which server the "connections" originate from, via server manager. They are coming from many different servers. I will run process manager on the "offenders" anyway. I do not build the servers so I will have to ask those that do if there is a possibiity that they have been cloned. (By cloning you do mean with a program like Ghost, right?)

 

[Highland]

Had a quick look at our servers to "see" what was using the IPC$ shares on them.....the IPC$ share is described by MS as an Admin share and I believe this means it should be available for the connection of printer shares, remote admin (performance monitor, etc..) and Remote Backup packages etc...anyway, looking at our servers it also appears to be used by "Citrix" apps....most of the connections using IPC$ were "used" by Citrix Server Admins...

I also found this as part of an article by MS....

Server Manager will show the connection user as the user's name while the share connection is maintained. If the user disconnects from the share, then the IPC$ connection user will change to the user's computer name. At this point, any user at that computer attempting to connect will have to authenticate with the server. Although the IPC$ connection is still intact, it is not associated with any user name or credentials, so it does not pose a security risk.

Do you have any software running on the Citrix Server which is acting as a "license manager" for a published app....???

 

[kgouldsk]

Highland, if you're seeing admin usernames, they're probably running the mfadmin tool and are connected to the other servers. IPC$ is like a non-specific connection to a server. If I do a
net use \\myserver /user:mydomain\kgould

to a server, it creates an IPC$ connection. This ensures that all further operations I perform against the server use that specific security authentication because an existing connection will always be used. Incidentally, you can create a SEPARATE security identity against a server by doing something like:
net use \\myserver.mydomain.com /user:mydomain\adminid

then you'd look like this:
net use
Status Local Remote Network
---------------------------------------
OK \\myserver\ipc$ Microsoft Windows Network
OK \\myserver.mydomain.com\ipc$
Microsoft Windows Network

Then you can reference the server in one context and get admin privileges, and the other and get regular user privileges. This won't work for all tools, since they don't always allow fqdn, but works for some circumstances. Totally off the topic, sorry, but educational nonetheless.

 

[Highland]

Yeah the connections are being generated by Citrix Server Admin in our environment, I was just highlighting the fact that Admin tools etc, use the IPC$ share to provide functionality on each server.....I'm not familiar with any of the "tools" provided by Metaframe XP (ie by specifying the servername rather than username within Server manager)and was wondering if any of these could be associated with the use of the IPC$ share.....????

Cheers....!!!!

 

[jdemmi]

Ok....here' is the latest.

SEVER A has a "connection" in Server Mgr as follows:
Connected Users Computer Name Time Idle
USER1 Server B 00:11 00:11

The resource "open" is IPC$.

So I login to SERVER B and view PROCESSES open by USER1 via Task Manager. He ONLY has WFSHELL, USERINIT, NDDEAGNT and the published app open. And before you attempt to go there do not blame it on the app. This happens FROM\TO all of our Citrix boxes regardless of the application(s) which run on them.

Any more ideas?

 

[Highland]

Are your servers load balanced....???? Do the connections you see under IPC$ relate to the load balancing....????

This is from an article on thinnet, not sure if it will help but it does have info on the processes you are seeing.....

USERINIT.EXE is the process spawned from WINLOGON.EXE
after a valid username/password & domain are supplied
to the GINA! This is the process responsible to
creating the users desktop (in normal NT) and other
published app type stuff under TS or MF ! You need
this process - do not remove it unless you want
trouble

NDDEAGNT.EXE on the otherhand is what is says - a
Network DDE Agent and can be removed is required, you
can remove it (or change NTFS perm's to stop is being
executed) as an experiement to see if it muck's inter
process comm's up with any apps - if so turn it back
on, otherwise leave it! The less you can run on a TS
the better..........

 

[jdemmi]

Yes the server are load balanced, but in my example above, SERVER A is NOT load balanced with SERVER B.

 

[Highland]

OK....!!

I think this has to do with the Event Log on the servers.....I had a look at what was happening with the IPC$ on our servers and it appears to be similar to the situation you have...IPC$ was opening for users I really didn't expect to see having access to the server...just shows what you miss...???!!!

Anyway I noticed a couple of users accessing the share and couldn't understand what resource they were using, all our printer shares, home directories etc are not defined on the Citrix Servers.....checked event viewer/security and the logon for each user was being recorded...????

If you check under User Manager / Policies / Audit ...this lists the events being recorded (on all servers in the Domain I think..!!!).......so I think that what's happening is that for every logon the actual logon server opens an IPC$ connection to each server in the domain in order to "pass" the info to each event log.....?????.....just checked...and that's not correct, some servers seem to record each logon and some don't...I'm not sure why...!!!! OK more investigation...........