Q-Logic FC card fails and MS update KB972270

[NeilChak]

Update KB972270 appears to cause a hardware fault in Q-Logic Fiber Channel cards in IBM 8843 blade servers. We had two failures using win2003-R2. System log 3 failovers/s of HBA connection until the card fails.

Solution is to replace the card,
Remove the update and block the update from installing.

 

[itsp1965]

Have you contact Qlogic support regarding the issue?

 

[NeilChak]

Q-logic does not support the cards. IBM support does so they know. I posted this so that others might be able to find out earlier than we did.

 

[itsp1965]

I understand that but since they are OEMed from Qlogic maybe you can try their support teams also as they may have already seen this issue on their equivalent card.

 

[NeilChak]

I submitted this info to q-logic's support site.

 

[NeilChak]

Hello,

Thank you for contacting QLogic Technical support. The cards that you have in your blade servers are IBM OEM HBAs. QLogic HBAs that use IBM approved drivers, BIOS/Firmware or software, or HBAs with model numbers that include -IBM are IBM approved boards. These HBAs and drivers, BIOS, and software require special approved support through IBM. Please contact IBM regarding support for these configurations.

For IBM technical support within the US, please call: (800) IBM-SERV
For IBM technical support outside the US, please visit IBM's Support Web Site.

http://www.ibm.com/support/

If you have any questions or need assistance with standard QLogic software or hardware, please let us know.
[Date] 2010-01-20 07:40:10 PST

 

[NeilChak]

I contacted Microsoft and they are working on this! Yea!!
Here is my info that I sent them.

Hi Naveen,
I'll relate the entire story to you.

1. We came in Monday morning (11 Jan) and a server (ARCGIS) windows 2003-R2 32bit IBM Blade type 8843 was down. When we tried to boot the server we got the message that the C drive was corrupted. The C drive is a Fiber Channel Raid 5 LUN on IBM DS4800 connected by the Q-Logic card and Q-Logic pass though and Brocade Switches.

2. After several miss steps we replaced the q-logic card. Then the C drive became bootable and the system booted up just fine. Also saying that a reboot had been necessary to install updates.

3. Looking at the system log to find the reason for failure we found 3 messages were being generated per second about the q-logic card failing over.

Below is one such message:
==========================
Event Type: Warning
Event Source: ds4dsm
Event Category: None
Event ID: 801
Date: 1/13/2010
Time: 5:15:31 PM
User: N/A
Computer: ARCGISPROD
Description:
Failover succeeded to depds4800:1.
Data:
0000: 00 00 00 00 03 00 52 00 ......R.
0008: 00 00 00 00 21 03 04 80 ....!..?
0010: 21 03 00 00 00 00 00 00 !.......
0018: 00 00 00 00 00 00 00 00 ........
0020: 00 00 00 00 00 00 00 00 ........

4. Looking at Windows Update history we noticed two new updates KB972270 and KB955759. We removed KB955759 which required a reboot.

5. The problem persisted, we continued to see 3 messages per second generated. We removed KB972270 after which the messages stopped.

6. We went to our other servers IBM 8832 and found the update KB972270 but no messages in the log. We ran out of time before checking all the servers and had to move on to other events.

7. We came in next Monday (18 Jan) morning and another server GISDATA1 was down (IBM 8843). When we attempted to boot the server it complained that it's c drive was corrupted. Again the c drive is a Fiber Channel Raid 5 Lun.

8. We replace the Q-Logic card and removed KB972270 and it has worked since.

9. Here is the information about the Q-Logic Card and Blade server, The Q-logic card is shown as the “Expansion Card”

Bay(s) Module
Name Manuf.
ID Machine
Type/Model Machine
Serial No. Hardware
Revision Manuf.
Date Part
Number FRU
Number FRU
Serial No. UUID
9 GISDATA1 IBM 884325U 99GZP74 6 1206 39R8650 31R3249 ZK126863R12W 40A3 2B7C 3230 B601 5153 0014 5E36 8D9A
9 Expansion Card IBM n/a n/a 3 0806 32R1778 26K4859 ZK11JZ62R1H8 7C83 0AE1 A5D9 11DA B4AD A0A1 08DE 2E65

10 ARCGIS IBM 884325U 99GZR13 6 1206 39R8650 31R3249 ZK126863R1BD 4014 AA9D 3030 B601 E3D0 0014 5E36 8C7C
10 Expansion Card IBM n/a n/a 3 0806 32R1778 26K4859 ZK11JZ62R17W DB16 C701 A5CE 11DA B455 8B30 4802 915F


I hope this information is of assistance and if I can help in any way I would be glad to.

Thank you
Neil Chakrabarty


-----Original Message-----
From: Microsoft Security Response Center [mailto:secure@microsoft.com]
Sent: Wednesday, January 20, 2010 1:53 PM
To: Chakrabarty, Neil
Cc: Microsoft Security Response Center
Subject: RE: Security Update KB972270 [np]

Hello Neil,

Thank you for reporting this issue to us. However, could you please give us additional details? We are trying to investigate how our software patch for MS10-001 (KB 972270) could cause a hardware fault.

Could you please elaborate on how you reached this conclusion?

Thanks,
Naveen Palavalli

-----Original Message-----
From: Chakrabarty, Neil [mailto:Neil.A.M.Chakrabarty]
Sent: Tuesday, January 19, 2010 11:18 AM
To: Microsoft Security Response Center
Subject: Security Update KB972270

Update KB972270 appears to cause a hardware fault in Q-Logic Fiber Channel cards in IBM 8843 blade servers. We had two failures using win2003-R2. System log shows 3 failovers/s of Q-Logic HBA connection until the card fails.



IBM Support replaces the cards when they fail but it took us a while to figure out the reason for the problem.



Thank you

Neil Chakrabarty

WV DEP

http://www.dep.wv.gov

304-926-0499x1626.