Many I/O errors on IBM LTO-2 drives

[hdvries]

Almost one year ago we installed a STK L700 library with IBM LTO-2 drives attached to our Legato NetWorker server on AIX. Library and all tapedrives are independently attached to the server and to several storagenodes using SAN connections.
From the beginning we suffer all kinds of problems in this environment, such as I/O errors, Connection timeouts, tapes which are not unloaded etc. We put a lot of effort in trying to solve these problems. We installed the highest level of firmware and drivers in the LTO drives, SAN (Brocade) switches and in AIX. Most of the problems disappeared, but we are still suffering I/O errors. During backup writing an I/O error occurs, the tape is set to "full" by NetWorker and the backup continues with another tape. It happens on different storagenodes (all AIX), but not on the server. It also happens on all available devices in the STK library. In the errpt of the AIX system we see no errors at all. Any suggestions ???

 

[Pachtva]

Hi.

This is a know issue with the AIX systems.. IBM can help you..

 

[hdvries]

Thanks Pachtva,

but can you be more precise. Is it a problem in the AIX tape drivers or in the IBM LTO-2 drives?

 

[Pachtva]

Well..

I have the same problem with a legato software and a L20 with 2 LTO Seagate Tape Drives. In the SAN have a windows, unix and AIX OS.

The problem its only present with the AIX OS v. 5.1, the others AIX OS have a different version a does not present this problem.

Support of Software IBM tell us that this problem its present in this version (v. 5.1) and the customer tell that change it.

And the problems stop. I dont know which version load the IBM engineers.

 

[hermxl1]

Hdvries, I'm curious to know whether or not you see a high amount of I/O traffic at the switch when backups are not running? I'm having a similar problem with LTO drives and Networker, but with Sun and Windows servers and storage nodes.

We think our HBA (Emulex) chatter is causing enough noise on the SAN to disrupt Networker when reading tape labels and this causes the tapes to be mark full and the save sets suspect.

Thanks,

Herm

 

[Pachtva]

Do you have the sense data of the problem? with this we can verify in wich part its present the problem

 

[embeegee]

We have Both IBM LTO 1 and 2 drives they are connected to different Sun Servers.
The LTO 2 is fine and runs prefectly the LTO 1 however follows the pattern above I/O errors and marking a tape full after having only a few hundred kb written to them.
Both Sun servers were running the latest patchset from Sun!
IBM suggested that we run the built in 3581 verify utility.
On the LT0 1 it failed on the LTO 2 the very same tape passed!
Following IBM's insistance that both autoloader and drive firmware be the very latest we loaded the latest updates.The result was the same.
IBM have agreed to replace the complete unit due on the 8th Dec.
I will let you know how it goes

 

[embeegee]

Solved !!!!

This was a faulty drive (Although the vendor took some convincing)
Installed the new drive and legato is now writing to the full capacity of the tape.
The key to working this out was the drive's internal tape verify utility (drive diags on this 3581) the tape failed the test on the faulty drive. However it passed with no problem on our LTO 2 3581.