invalid username password error

[madhuusa]

Team, We are running
LL 9.5 SP1 with sql server 2000 and EFS in a seperate machine. The front end is 2 machines running 3 instances each of LL.

Of late we are getting this error called invalid username/password when users try to access Livelink. We were in a prior monthly patch but put the latest May 2008 monthly patch after this problem started coming.

2 years back we got the same problem and we were given a patched llodbc.dll. But now the same error is occurring and the dll file size in latest monthly patch remains the same.

Anybody here having the same issues like this. Or any ideas why this is happening?

The error goes away once we resrart the livelink services only to come back in a week or less again. Advance thanks for your help.

08/01/2008 01:12:04 [1997433527] 0000804541: KSqlCursor::Open(2000,'select a.*,b.OwnerID GroupOwnerID,b.Type GroupType,b.Name GroupName,b.UserData GroupUserData from KUAF a,KUAF b where a.Type=:A1 and a.SpaceID=:A2 and a.Name=:A3 and a.GroupID=b.ID') -->'SUCCESS'
08/01/2008 01:12:04 [1997433604] 0000804542: KSqlCursor:repare() --> 'SUCCESS'
08/01/2008 01:12:04 [1997433708] 0000804543: KSqlCursor::Bind(3,{0,0,'Appsmonitor01'}) --> 'SUCCESS'
08/01/2008 01:12:04 [1997433768] 0000804544: ODBC diagnostic error: 1 ODBC error code: 0 ODBC state: 08S01 ODBC error: [Microsoft][ODBC SQL Server Driver]Communication link failure
08/01/2008 01:12:04 [1997433794] 0000804545: KSqlCursor::Execute() --> 'Could not execute cursor.'
08/01/2008 01:12:04 [1997433841] 0000804546: KSqlCursor::Close() --> 'SUCCESS'
08/01/2008 01:12:04 [1997433871] 0000804547: KSql::Execute(...) --> 'Error executing an Sql statement.',0 records,[sec: 0 msec: 0]
08/01/2008 01:12:04 [1997433900] 0000804548: KSqlCursor::Open(2000,'select a.ID,a.OwnerID,a.Type,a.Name,a.UserData,b.*,c.OwnerID GroupOwnerID,c.Type GroupType,c.Name GroupName,c.UserData GroupUserData from KUAF a,KUAFUser b,KUAF c where a.Type=:A1 and a.SpaceID=0 and a.Name=:A2 and a.ID=b.UserID and b.GroupID=c.ID') -->'SUCCESS'
08/01/2008 01:12:04 [1997433982] 0000804549: KSqlCursor:repare() --> 'SUCCESS'
08/01/2008 01:12:04 [1997434033] 0000804550: KSqlCursor::Bind(2,{0,'Appsmonitor01'}) --> 'SUCCESS'
08/01/2008 01:12:04 [1997434074] 0000804551: ODBC diagnostic error: 1 ODBC error code: 0 ODBC state: 08S01 ODBC error: [Microsoft][ODBC SQL Server Driver]Communication link failure
08/01/2008 01:12:04 [1997434098] 0000804552: KSqlCursor::Execute() --> 'Could not execute cursor.'
08/01/2008 01:12:04 [1997434136] 0000804553: KSqlCursor::Close() --> 'SUCCESS'
08/01/2008 01:12:04 [1997434164] 0000804554: KSql::Execute(...) --> 'Error executing an Sql statement.',0 records,[sec: 0 msec: 0]
08/01/2008 01:12:04 [1997434199] 0000804555: KConnect::ErrLog --> [10106,5],'Invalid username/password specified.'
08/01/2008 01:12:04 [1997434228] 0000804556: KUAFUser::Get(Appsmonitor01) --> 'Invalid username/password specified.'
08/01/2008 01:12:04 [1997509275] 0000804557: CAPI.Log: Fri Aug 01 01:12:04 2008 - 55484628 Func='ll.login' Timing: 0.078 0.079 0.078

 

[appnair]

Don't have your hopes too high on this.I chased this with OT support for long but it is very difficult for even them to re-create this.I even gave them a way to re-create it.I would notice this with our livelink returning back from work after a friday.Clicking succesively the "invalid" screen will sometimes start it back up.I once saw this in builder trying and doing some calculation against the kstate table,nobody even gave me what that table stood for.As a workaround I kept a pinger that would hit livelink very 30 minutes or so.

If you find thi out please let me know

Well, if I called the wrong number, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

 

[madhuusa]

Hello AppNair,
That's surprising to know.

Ours is a heavily used system with 10 custom modules + half a terabyte of data with 4000 users.

This error came like 1.5 years back and we patched it with a odbcdll.jar and the problem vanished.

Like 2 months back it came once and then last 2 weeks. Users all over the world were stuck big time and it brought their teams to a virtual halt as we did not expect this to happen and so toned down the 24/7 support a bit. Can't afford one more downtime for this

Can you give more details. Are you in 9.5 SP1. Is workflows being used heavily. Any custom application writing complex sql via CAPI?

So far in past 3 days there has been no issues since we put the May 2008 patch. But the usage has been low due to the friday and the weekend.

How many times this error has come to you in past say 6 months. At what frequency? What do you do to fix it? Just restart of Livelink services enough?

 

[appnair]

I don't remember most of the details.9.5 SP1 is almost at the end of the maintenance spectrum..But I could almost remember that this was some network related error.Something like LLserver is running and then there is ahuge lull period like hours in between(Friday evening to Monday morning).Also on saturday night at 2.00 am the DB was downed for cold backup,Maybe it is this loss of oracle service that was causing it.It was never a big problem for us.

Some tools exist that look at SNMP traces.Have you seen a product called the monitor(do not get it confused with the performance analyzer).

My gut feeling s that it has nothing to do with livelink but something going wrong with the network.In my install the oracle traffic was in a separate subnet so that things were sooth for livelink as well.

Well, if I called the wrong number, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

 

[ggriffit]

Having had the same issue with a few customers the response I've had from OT is that what is happening is that Livelink loses connectivity to the database for some reason and so LL drops the connection, however after thread + 1 connection attempts LL will repoll the database and reconnect properly.

There are patches, such as the dll referenced above which resolve the issue by polling on every request, however they can be just as messy for several reasons including :

1. Overhead of polling on each request to the DB
2. As soon as the DB is visible after a loss of connectivity it will reconnect, this would include a DBA bringing up a DB for maintainence or investigating a serious issue on the DB.
3. If you don't 'see' the outage it may mask a serious underlying issue in your setup whether that be Network or DB related.

As AppNair highlighted the issue is normal a Network related issue so I'd start there.

Greg Griffiths
Livelink Certified Developer & ECM Global Star Champion 2005 & 2006

http://www.greggriffiths.org/livelink/

 

[appnair]

Greg explain this if you can for my benefit

"however after thread + 1 connection attempts LL will repoll the database and reconnect properly."

So on a normal 5 thread scenario what does this translate to.

I issued a request it hit thread5 and it got a "invalid username password"

I refresh my browser it may now hit thread2.is it going to re-poll now...That means that LL is writing that info somewhere may be to a file.It can't do a capi transaction because it would have the same fate.

Very weird like I said I spent a lot of time trying to chase it with no avail





Well, if I called the wrong number, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

 

[madhuusa]

Greg, AppNair,
The problem is fixed. Meaning, this is what happened.

An odbc dll was released some in middle of 2006 (for 9.5 SP1) to take care of connection leak. It was not in monthly patch but was released as a seperate patch.

Now between Dec 2006 and Dec 2007, if ANY monthly patch was installed AND any of jan 2008 till may 2008 monthly patches was NOT installed, then this problem will re-occur.

Because between dec 2006 and dec 2007, the connection leak patch was overriden by other monthly patch fix.

Now will this cause invalid username every day. no. but it will come periodically and if you are unlucky consecutively on several days before vanishing again.

We were in Dec 2007 patch as we were planning to migrate to 9.7 SP1 and also due to our own oversight miss. So this error came again.

With install of May 2008 patch, this defect is now closed and for past 3 days it is doing fine.

If support had issued an security alert on Jan 2008 about this, it would have been awesome.

Anyway's all is good now. Atleast till we see next time this error

 

[madhuusa]

Just a quick update on this.

9.5 SP1 is in retirement mode. Meaning no more monthly patches, no hot fixes, no bug fixes, etc.

9.7 SP1 has potentially eliminated this issue since it has more robust connection handling mechansim.

Of course, if your odbc connection keeps dropping, then tracking the connection also will give helpful hints.