Mercator Integration with JMS

[mlapse]

Hi,

We use mercator maps to write to a JMS queue. When we ran the system in production, we found that we were getting errors in the launcher file like (Target Not Available)

THis error was not reported for all messages, only some. I believe it could be because of the idle connections becoming invalid and mercator trying to use them.

I need to know whether there is some way to simulate this. Is there some timeout limit for Weblogic JMS connections?

Also there are parameters in the mercator.ini file. However since I cant simulate this condition (on the test machine, it seems to work no matter when i try to put the files and no matter how many files I have)

Please help!

 

[janhes]

The recommended .ini file settings to avoid reuse of invalid connections are
IdleJMS=10
;SLimJMS=0
;HLimJMS=0
KeepJMS=0
;MinJMS=0

I also have failures occasionally and I think I have tied them down to jdbs connections to the JMS persistant storage. I think the weblogic connection pool is too small when a large number of messages are processed.

 

[BocaBurger]

It might help to put a -TE on the JMS adapter command line (if you are on a recent version) to find out what is happening. What are the differences between test and production?


BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[mlapse]

jahnes and boca burger,

Thanks. But what do you do when you recieve these errors? Do you restart the service? Why dont you set the KeepJMS parameters also?

I would like to prevent these errors and if they cannot be prevented, at least ensure that the messages get retried or processed some way.

 

[mlapse]

Also would the retry switch being on help with processing the messages?

 

[mlapse]

janhes,

after a whole lot of testing today, I think you are correct. I processed a whole lot of files (about 4000) and there was just one error. All other files were processed successfully.

The funny thing was that the error was a return code of "9" - Target not available and the adapter return was "-305". This error is the same I get when Weblogic is down and mercator is running and trying to put msgs to JMS!!! But Weblogic was up and running. I renamed the file and it immediately got processed. This file failed in the middle of all other files for no apparent reason. I checked the number of open connections to JMS and found that there were only 5 which were being reused.

So now what I need to know is how do you take care of the errors you still get? Do you use the retry option? if I retry the file after another 10 seconds (in addition to the mercator.ini) settings will it work ?

thanks

 

[janhes]

Yes, I've got the same problem. I can't get any info from the JMS trace about this. I did get a diagnostic patch from support but was unable to recreate the problem on development. The only indication of any problems was a suggestion that I was getting connection failures to the Weblogic persistent storage on an Oracle database. I an currently investigating this.
IO have always found that changing the timestamp on the files allows them to be reprocessed but this is not an ideal solution.
Let me know if you find anything else.

 

[mlapse]

We are using a file based storage for our JMS queues and not a database. So I think this issue is probably because of the some connectivity not being there (as you pointed out).

I have been able to recreate this scenario in test. I pushed a whole lot of files for the JMS map as well as other maps and somehow the JMS map failed with an adapter return of -305.

I will test the retry option today and let you know if it worked.

 

[BocaBurger]

The 305 error is a memory error for the JM service. There are settings in the dstx.ini file from 7.5.1 on to adjust this.



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[janhes]

Is there any way to overcome this at 6.7.1?

 

[BocaBurger]

Upgrade. I tried this with a 6.7.1 patch for one customer, but there seems to be more needed than just the files I got. Customer was on UNIX, so they installed an 8.0 Event Server, upgraded the few maps that used JMS, and now works OK.

BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[janhes]

I have had an event server patch along with a new m4jms.jar to overcome a problem where the event server crashed if I logged out the user on the server running the event server.
This started utilising these parameters in the mercator.ini file
[JVM Options]
option1=-Xrs

[JNI Layer Trace]
Switch=on
File=E:\MERCATOR6.7\mercjpiX.log

I have since added to JVM Options

option2=-Xms200m
option3=-Xmx300m

to my development system. It had no detrimental effect though it increased the VM size by around 200 mb.
Unfortunately, I have been unable to replicate the 305 error on development so I think I'll have to try this in production to see if it cures the problem.

 

[mlapse]

janhes,

Have you been able to resolve this issue? I tried contacting Mercator support and they said that this issue had been reported to them but they had no solution for it yet.

They may be able to provide a JMS debug trace but it doesnt seem to provide much detail about the error.

 

[janhes]

I'm putting the change into my production system tomorrow. I'll let you know what happens.

 

[mlapse]

janhes,

Did it work?

I received some fix from mercator support. the gave me a new mercjpi.dll and asked me to change the settings in the ini file. the option 1 and option 2 were to be made -Xms200m and -Xmx300m.

However on restarting mercator, the event server in the management console showed me that the server was in Stopped status. Couldnt understand why though? Did you face this same issue? What did you do about it ? What did you send to mercator support to get them to fix it?

thanks for your help.

 

[janhes]

mlapse,
I put the change in last night but it made no difference. The only effect was that the vm size increased by 200mb.
I think that this must be another problem. I have noticed that the -305 error does not result in a jms connection failure in the Management Console statistics so it must not be getting as far as trying the connection.
What version of Weblogic are you using - we've got 6.1.

 

[janhes]

Ignore the bit about JMS connection failures not showing in the Management Console. They are now.

 

[mlapse]

janhes,

We made a change to our JMS system a couple of days back. Earlier we were processing about 60000 messages in a day. Each message was 2KB in size.

We grouped some of those messages together before putting to the JMS queue. The number of messages put to the queue dropped to about a 1000 a day. Each message was of the size 4KB.

The key thing is that after this change I have not seen any more of these errors at all.

dont know what changed but the problem has been taken care of by doing this. i have asked IBM support to check this out asap.

 

[janhes]

mlapse,
Do you run multiple copies of the map putting messages on JMS and do you have tracing set on the JMS adapter?
I have trace set on mine and call the JMS map from 2 different maps. I'm wondering if there is occasionally conflict on the trace file when there is a high volume of messages being processed. I think I'll try removing the trace and see if it makes any difference.

 

[mlapse]

my JMS setup runs when called from one map only. There is no trace file set and the log files are also unique.

IBM support is checking and they are trying to figure out whats wrong. Will keep you posted.