We have recently ran into a problem with a COM+ application that we have
been developing. The application consists of a set of Visual Basic front end
clients that use SOAP (over WAN, through firewall etc.) to talk to a set of
Visual Basic COM+ components on two load balanced mid-tier servers, which
talk to a MS SQL Server 2k database.
When we rolled it out we started getting the following message on the front
end clients and the application became unusable:
Method '~'of object '~' failed
Error Code: -2147023898 (Invalid access to memory location.)
The major problem is that we only have 7-8 clients machines using it at the
moment. As soon as this message starts appearing about 90% of the clients
cannot do anything else in the application without getting the same error
message. It's also happening on average about once a day and so far appears
totally random.
Here is some more info that I think might help to track this problem down.
We have a set of seven business logic COM+ components that are added to a
Server Application in component services and one database COM+ component
that is added to a Library Application (all of which we developed
ourselves). One of the server objects is setup to handle all of the
transactions (classes set to RequiresNewTransaction) within the system and
makes calls out to the other server objects (classes used by main
transaction object set to UsesTransaction). Not all of the methods are
wrapped in a transaction, just the critical ones. All of the components are
setup as Apartment Threaded, Unattended Execution and Retained In Memory. We
haven't changed any of the default settings for the server and library
applications in components services i.e. applications created, objects are
added, activation strings entered and security is setup.
Also, all of the mid-tier machines, database machines and front end clients
have the latest services packs etc. installed.
We started to test a few things when this started happening. When it
happened the other day we pulled the network cable out of one of the
mid-tier machines and the application started working again. We then plugged
it back in and it continued to work until the next day. When it happened the
next day we tried unplugging the same mid-tier machine but that didn't work,
so we tried the other one and the application returned to normal.
We think that the reason this is happening is that one of the mid-tier
machines is getting clogged up somehow and taking this "bad" mid-tier
machine out of the equation sends all the client requests to the "good"
mid-tier machine and gets them working again. We are currently doing tests
over the next couple of days that involve taking the load balancing out of
the equation and running the application on only one mid-tier machine (a
different one each of the next two days).
We also just noticed during today's test (just the one mid-tier machine
active) that shutting down the server application from component services
returned the application to normal.
We have also noticed the following error in the application logs on the
mid-tier servers:
String message: Session idle timeout over, tearing down the session.
I've read up a bit about this message and it doesn't seem to be a problem -
just a warning that the server application in component services is shutting
down due to inactivity but it still might point to a possible problem.
Has anyone come across this sort of problem before or have you some advice
on how we should go about tracking down a solution?
The possible culprits we have come up with so far are as follows (some we
know are probably not to blame but we aren't ruling out anything):
1. Load balancing. (At the moment one of the mid-tier machines seems to be
getting a lot harder and more frequently than the other one but this could
be down to other applications e.g. Patrol, running on one of them and the
small amount of clients we have at the moment)
2. Memory leaks in application/components i.e. references to objects not
always being set to nothing (e.g. if error routine is called). I'm not 100%
sure how a SOAP enabled application handles objects that are not implicitly
closed.
3. Something to do with SOAP and the connection from the client to the
server. Could be that the connection is being kept open when it shouldn't or
being closed when it shouldn't, leading to problems on the server. I don't
have an extensive knowledge of SOAP based applications, so I'm unable to
make a call on this one.
4. Firewall. I don't think this one is very likely but not ruling it out
yet.
5. Hardware issue. Don't think it's this either but still a possibility none
the less.
Has anyone got anything they could add to this list or more information on
them?
I think the big clue to our problem is the -2147023898 (Invalid access to
memory location) error but I from what I have read I haven't been able to
figure out if a) this is the root problem or if it's caused by another
problem, and b) what problem with COM+ does this error point to?
Any help you could give would be greatly appreciated. If you would like more
info then just shout.
Regards
Steve
been developing. The application consists of a set of Visual Basic front end
clients that use SOAP (over WAN, through firewall etc.) to talk to a set of
Visual Basic COM+ components on two load balanced mid-tier servers, which
talk to a MS SQL Server 2k database.
When we rolled it out we started getting the following message on the front
end clients and the application became unusable:
Method '~'of object '~' failed
Error Code: -2147023898 (Invalid access to memory location.)
The major problem is that we only have 7-8 clients machines using it at the
moment. As soon as this message starts appearing about 90% of the clients
cannot do anything else in the application without getting the same error
message. It's also happening on average about once a day and so far appears
totally random.
Here is some more info that I think might help to track this problem down.
We have a set of seven business logic COM+ components that are added to a
Server Application in component services and one database COM+ component
that is added to a Library Application (all of which we developed
ourselves). One of the server objects is setup to handle all of the
transactions (classes set to RequiresNewTransaction) within the system and
makes calls out to the other server objects (classes used by main
transaction object set to UsesTransaction). Not all of the methods are
wrapped in a transaction, just the critical ones. All of the components are
setup as Apartment Threaded, Unattended Execution and Retained In Memory. We
haven't changed any of the default settings for the server and library
applications in components services i.e. applications created, objects are
added, activation strings entered and security is setup.
Also, all of the mid-tier machines, database machines and front end clients
have the latest services packs etc. installed.
We started to test a few things when this started happening. When it
happened the other day we pulled the network cable out of one of the
mid-tier machines and the application started working again. We then plugged
it back in and it continued to work until the next day. When it happened the
next day we tried unplugging the same mid-tier machine but that didn't work,
so we tried the other one and the application returned to normal.
We think that the reason this is happening is that one of the mid-tier
machines is getting clogged up somehow and taking this "bad" mid-tier
machine out of the equation sends all the client requests to the "good"
mid-tier machine and gets them working again. We are currently doing tests
over the next couple of days that involve taking the load balancing out of
the equation and running the application on only one mid-tier machine (a
different one each of the next two days).
We also just noticed during today's test (just the one mid-tier machine
active) that shutting down the server application from component services
returned the application to normal.
We have also noticed the following error in the application logs on the
mid-tier servers:
String message: Session idle timeout over, tearing down the session.
I've read up a bit about this message and it doesn't seem to be a problem -
just a warning that the server application in component services is shutting
down due to inactivity but it still might point to a possible problem.
Has anyone come across this sort of problem before or have you some advice
on how we should go about tracking down a solution?
The possible culprits we have come up with so far are as follows (some we
know are probably not to blame but we aren't ruling out anything):
1. Load balancing. (At the moment one of the mid-tier machines seems to be
getting a lot harder and more frequently than the other one but this could
be down to other applications e.g. Patrol, running on one of them and the
small amount of clients we have at the moment)
2. Memory leaks in application/components i.e. references to objects not
always being set to nothing (e.g. if error routine is called). I'm not 100%
sure how a SOAP enabled application handles objects that are not implicitly
closed.
3. Something to do with SOAP and the connection from the client to the
server. Could be that the connection is being kept open when it shouldn't or
being closed when it shouldn't, leading to problems on the server. I don't
have an extensive knowledge of SOAP based applications, so I'm unable to
make a call on this one.
4. Firewall. I don't think this one is very likely but not ruling it out
yet.
5. Hardware issue. Don't think it's this either but still a possibility none
the less.
Has anyone got anything they could add to this list or more information on
them?
I think the big clue to our problem is the -2147023898 (Invalid access to
memory location) error but I from what I have read I haven't been able to
figure out if a) this is the root problem or if it's caused by another
problem, and b) what problem with COM+ does this error point to?
Any help you could give would be greatly appreciated. If you would like more
info then just shout.
Regards
Steve
