Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations SkipVought on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

IBM MQ - Message consumption options and performance issue

Status
Not open for further replies.

try2006

Programmer
Dec 17, 2006
1
0
0
CA
Overview

We have a scenario in our application to explicitly consume 10,0000 messages from a queue having 1,00,000 message (Queue depth). Basically we are expecting a solution either in Java or C++ that can consume these messages in a lesser time. Right now with Java it takes about 32 minutes and with C++ it takes about 42 minutes. Please note that these 10,000 messages are to identified based on its (JMS)CorrelationID header value. Actually these correlationIDs have a pattern. For example, consume all messages with the (JMS)CorrelationID that contains the String "A" [In this case the 10,000 messages will have the (JMS)CorrelationID as 000000000000000000A1[0000 - 9999]

Approach

Java

We create a QueueReceiver on a QueueSesion using message selector technique. For example we use the following statement.

Queue queue = //Get the required queue
String selector = "JMSCorrelationID LIKE '%A%'"
QueueReceiver queueReceiver = queueSession.createReceiver(queue, selector);
Message m;
TextMessage msg;
while ((m = queueReceiver.receive(1000)) != null) {
msg = (TextMessage)m;
}

C++

//Open Queue etc
//Create a Vector with the require correlationIDs to consume
vector<string> corrVec;
for ( int i = 10000; i < 20000 ; i++ )
{
char IntStr[10] ;
sprintf( IntStr, "%d", i ) ;
string integer = IntStr;
string str = "A" + integer;
string newstr = createCorrelationId(str);
corrVec.push_back(newstr);
}

// Iterating through the vector and construct a message with each CorrealtionID and consume
vector<string>::iterator iter;
string strCorrIdParam;
for ( iter = corrVec.begin(); iter != corrVec.end(); iter++ )
{

strCorrIdParam = *iter;
cout << strCorrIdParam <<endl;
char * respBuffer = new char[RESPONSE_BUFFER_SIZE];
msg.useEmptyBuffer(respBuffer,RESPONSE_BUFFER_SIZE);

//Prepare to GET and PUT the message
gmo.setOptions( MQGMO_WAIT | MQGMO_FAIL_IF_QUIESCING );

gmo.setWaitInterval( MQWI_UNLIMITED );

MQLONG buflen;
//Since we are NOT looking for a specific message, //set the message id and the correlation id to default values
msg.setMessageId( );

// Adding correlation Id to MSg
msg.setCorrelationId(ImqBinary(strCorrIdParam.c_str(),24));

//Get a message from the queue
string strResponseMessage;
if ( in_queue.get( msg, gmo ) )
{
if ( msg.formatIs( MQFMT_STRING ) )
{
char *msgData = msg.bufferPointer();
buflen = msg.dataLength();
char *buffer = new char[buflen + 1];
memcpy(buffer, msgData, buflen);
buffer[buflen] = '\0';
strResponseMessage = buffer;
delete [] buffer;
}
}
delete [] respBuffer;
}

Problem

First we used Java with the messageSelector approach and as we heard that this approach is not an efficient one, we moved on to C++ with a straight forward approach to look up for each specfic message. But this also did not meet our expectations.

Requirement(s)/Questions

1) Is really C++ way is better than the Java's message selector approach?
2) Is there a better way (in JMS or MQ API) to quickly consume a message based on its JMSCorrelationID either in C++ or Java?
3) Is there any way in MQ administration/Configuration options to allow consumption/delivery faster?
4) Is there any code clean up required in the above source code?
5) In JMS/MQAPI we also have Consumer, Browser classes apart from Receiver class. Will that improve performance?
6) Can we improve in anyway at client side? For example run a dedicated Thread/Process that only does this Job, avoid logging during this process, Close any other resources etc?
7) Is there an entirely different solution for this?

Any help is highly appreciated.

Thanks,
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top