LSI SAS drivers w/ RH EL 5

[JMCraig]

Hi Folks,

I've been fussing with what should be a fine piece of hardware (Supermicro MB w/ LSI SAS1068e RAID controller) and have not been able to get the drivers to work w/ a trial install of RH EL 5 (2.6.18-92.el5). The OS is installed and happy on two SATA drives, but the RAID array--composed of my expensive SAS drives--is invisible (fdisk doesn't list it).

I've got the RH 5 RPM available from LSI. (I tried Debian before without success [no luck building from source as guided by someone who knew how to do it] but I think I may have had the wrong version of the source--it built but the drivers wouldn't load.) I decided to try RH because there was an RPM available:

LSI's Downloads for 1068e Controller

The latest stuff from LSI seems to be version 2.6.18-53 (loading the RPM apparently created several directories in /lib/modules) while the balance of the stuff in /lib/modules is under 2.6.18-92.el5 (I don't know if the version number in the paths to the drivers [2.6.18-53] means the OS won't find them, but I'm assuming that might be so--I'm a DB guy so I'm beyond my knowledge here). The RPM loaded up fine but the /etc/init.d/fusion.mptctl service doesn't want to start (I get this error as part of the startup as well as when attempting it manually after bootup):

[root@superm]# /etc/init.d/fusion.mptctl start
Starting MPT Fusion (ctl) drivers:                         [FAILED]

Not sure what my next step should be.

How do I find any error log entry there may be? (Nothing seems to show up in /var/log/messages.)

Can I just copy the /lib/modules/2.6.18-53.el5 files into the /lib/modules/2.6.18-92.el5 directory (and maybe they'll work)?

Do I need to recompile from source for the particular kernel version I'm dealing with? (If so I'll have to find/hire someone to help with that!)

Any suggestions most gratefully accepted!


John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[Annihilannic]

Does lspci | grep mpt show any loaded modules for this device?

Can I just copy the /lib/modules/2.6.18-53.el5 files into the /lib/modules/2.6.18-92.el5 directory (and maybe they'll work)?

No harm in trying.

dmesg should display your boot messages, if the driver/module is loading you should see something MPT related in here.

Are there any documentats/installation instructions in the RPM? Try rpm -ql rpmname to list its contents and see if there are any likely candidates.

Annihilannic.

 

[JMCraig]

Does lspci | grep mpt show any loaded modules for this device?

Shows nothing w/ the grep mpt. w/o that, shows this:
02:00.0 SCSI storage controller: LSI Logic / Symbios Logic MegaRAID SAS 8208ELP/
8208ELP (rev 04)
Which is not the controller that the MB claims it has--and, I don't think it's what the LSI BIOS sees either (have to check that).

Can I just copy the /lib/modules/...

Didn't seem to help (darn it).

dmesg makes no mention, that I can see, of anything related to the LSI controller.

I did notice that there's an i2o driver (which I've encountered with Adaptec RAID controllers before) that is in the /lib/modules tree. Not sure if that could be causing problems or not.

Are there any documents/installation instructions in the RPM?

Nope. Just a bunch of things like this:
/lib/modules/2.6.18-53.el5/kernel/drivers/message/fusion/mptbase.ko.new
/lib/modules/2.6.18-53.el5/kernel/drivers/message/fusion/mptctl.ko.new
....
However, there is a file with some instructions in the tarball that includes the RPM. That file is seriously out of date (doesn't mention the fact that the tarball now includes drivers for RHEL 5--but it does). And refers to files that are not in the tarball.

Here's what it says:

=== Adding Pre-Compiled Binaries to an Existing Installation ===

Driver binaries are provided in 1.44MB images that are suitably
formatted to use as driver update disks for:
    Red Hat AS2.1, EL3.0, EL4.0 (i686 and ia64); EL3.0, EL4.0 (x86_64)
    SuSE SLES8, SLES9 (i386, ia64, x86_64);
and in RPMS:
    Red Hat  i686  : AS2.1, EL3.0, EL4.0
    Red Hat  ia64  : AS2.1, EL3.0, EL4.0
    Red Hat  x86_64: EL3.0, EL4.0
    Red Hat ia32e  : EL3.0 Quarterly Update 2, 3, 4, 5
    SuSE     i386  : SLES8, SLES9
    SuSE     ia64  : SLES8, SLES9
    SuSE     x86_64: SLES8, SLES9
RPMs should be used to upgrade the driver post-install.

The RPMs contain binaries for the install kernel and the latest two
released errata kernels for each distribution at the time the RPM
was created.  See the README_rpms.txt file for a listing of
the kernels supported by the RPM. The update disks should be
used for architectures not supported by RPM.

The driver disk image can be transfered to floppy disk with the
rawrite tool from dos, or the dd utility in Linux.  Here is the URL
for rawrite:
[URL unfurl="true"]http://www.tux.org/pub/dos/rawrite/[/URL]

Special Instructions for Red Hat installs: To use the driver disks,
follow the instructions from the previous paragraph to transfer
the image to a diskette.  Then when installation is complete, do
the following:
        # mount /dev/fd0
        # cd /mnt/floppy
        # ./post_install.scr
        # cd
        # umount /dev/fd0

To use the RPMs, the standard install, the erase and upgrade commands
may be used.  The driver source will be placed in your
installations RPM Base directory, i.e. /usr/src/redhat or
/usr/src/packages.  On the ftp site,
    ftp> cd 2.05.06
    ftp> cd mptlinux-rpms
download the appropriate tarball.

Which would be fine except that:
1. No floppy drive so I just tried the supplied RPM using the normal rpm -i approach.
2. Looking at the "update diskette" image which I dd'd to a spot on /mnt for inspection, the ./post_install.scr script does not exist.

The rest of the instructions provide some guidance on rebuilding from source (but it's a bit sketchy for a kernel novice [that would be me]).

But, it does appear that is what I need to try. (Rats.)






John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[Annihilannic]

Does lspci | grep mpt show any loaded modules for this device?

Oops! That was actually a typo, I meant lsmod | grep mpt, can you check that too please?

However, the lspci information was useful anyway... are you sure you really have the hardware you think you have?

Is the MegaRAID controller being detected and/or used? MegaRAIDs are a different family of controllers also manufactured by LSI (originally manufactured by AMI / American Megatrends, if I recall correctly, before they were purchased by LSI).

If you do indeed have an MPT type controller in your system, but it is not visible in lspci, I don't fancy your chances of getting it working!

Annihilannic.

 

[JMCraig]

1st, thanks for sticking with me on this!

I meant lsmod | grep mpt, can you check that too please?

Nothing shows up. (Unsurprising.)

However, the lspci information was useful anyway... are you sure you really have the hardware you think you have?

Good question! Somewhat oddly, the message from the RAID BIOS at bootup/POST is ambiguous with regard to MegaRAID vs. MPT:

LSI MegaRAID Software RAID BIOS version M1068e.01.08221427R
LSI Logic MPT RAID found at PCI Bus No.:02 Device No.:01

So, the line about the BIOS mentions both "MegaRAID" and the name of the controller model on the MB I ordered: 1068e. The next line mentions an MPT device at a particular bus address. Hmmm. Seems plausible that it looks like a MegaRAID 8208ELP to the OS's hardware detection. So, I'm going to try downloading that driver (since RH seems to want it) and see what happens....




John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[JMCraig]

OK. All the foregoing was way too complicated compared to what actually worked. It ended up that I did not use any additional drivers that didn't come with RH EL5 itself.

The original modprobe.conf looked like this:

alias eth0 forcedeth
alias eth1 forcedeth
alias scsi_hostadapter sata_nv

OK. So based on info in a driver diskette image's readme file (image downloaded from Supermicro's site), clearly the SATA RAID (nVidia chipset) is already listed and good to go (although I'm not using it as a hardware RAID); that means that the SAS controller has to be given a different alias (or I could skip loading the on-board SATA RAID driver). I decided to leave the sata_nv driver and add an entry for the SAS controller as: scsi_hostadapter1. I also discovered by looking at the directories in /lib/modules where the sata_nv.ko file was, and via modprobe -l that a whole bunch of megaraid drivers were already there, and, indeed, loaded! (Wish I'd figured that out sooner!)

So, I figured what if it just tries to use what's there? It worked! (Relief and disgust at how hard it was!) So, here's what got the SAS RAID arrays recognized:

In /etc/modprobe.conf added:

...
alias scsi_hostadapter1 megaraid

I added a corresponding install line to modprobe.d/modprobe.conf.dist:

...
install scsi_hostadapter /bin/true # this was already there
install scsi_hostadapter1 /bin/true # new line
...

Then ran depmod to rebuild any dependencies that might need to be changed. (I used the -A option which should only rebuild modules.dep if there were changes needed and it did rebuild so I assume this was a critical step.)

Rebooted and suddenly I have additional (and correct) devices listed by fdisk -l for the two RAID arrays I'd set up. Wow! What an ordeal....

I hope this helps someone else. (I have not figured out what the "Fusion" drivers from LSI are supposed to do compared to what is running now, and I still need to find a management program to run under Linux--haven't seen that yet on LSI's site--unless that's part of the fusion stuff, but at least I can use the hardware I paid for now!)

John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[Annihilannic]

Glad you got it sorted out.

The Fusion drivers are for the MPT ("Message Passing Technology") family of LSI SCSI controllers... it took me ages to figure out how to enable support for that in the Linux kernel because it's very inconveniently listed in a separate section to all the other SCSI modules in the kernel config.

Annihilannic.

 

[JMCraig]

So, Annihilannic, is it worth trying to get the Fusion package to load also? The 1068e's software on LSI's web site is the Fusion package and the 2nd line of the bootup BIOS message seems to suggest it would work.

Any guidance on whether I should proceed with trying to get that to load--would that give me better performance?



John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[JMCraig]

What "section" of the kernel config are you referring to for enabling support for the Fusion driver, Annihilannic?

John Craig
Alpha-G Consulting, LLC

http://www.alphagconsulting.com

 

[Annihilannic]

I'm referring to the make menuconfig you run before compiling/building your own kernel, which is probably not relevant to your requirements.

I don't know about trying to load the MPT drivers too... it's possible that LSI are trying to use this MPT driver model for all of their hardware now as it means supporting only one API, but I can only guess.

Annihilannic.