iSCSI adapters, VIOS, p5, AIX5.3 1

[cccompton]

We will be getting some p550 boxes and are considering using FC1986 iSCSI TOE PCI-X adapters to connect to a NetApp storage appliance. I've been looking for a document on using iSCSI with AIX and in particular VIOS, but haven't found anything.

My questions are:
(1) Can I configure the iSCSI adapter in a VIOS LPAR? I have found the procedure for configuring a "normal" AIX LPAR, but am worried it may not work with the limited VIOS commands.
(2) Once the iSCSI adapter is configured, what do the LUNs served up by the NetApp device look like to the AIX/VIOS OS? Are they discovered using "cfgmgr"? Are they "seen" as "hdiskN"?
(3) Can I use these iSCSI LUNs in the VIOS to set up Virtual SCSI devices for the client LPARS?

Any specific procedures or links to documentation would be appreciated.

Thanks,
CC

 

[cspilman]

1 - Have not seen this done on VIO Server but should work if you go the OS level by issuing oem_setup and then going through the normal procedure to setup the ISCSI initiator and LUNs.

2 - The LUNs are seen as regular hdisks to the AIX client. You can see them by running cfgmgr.

3 - Don't know about this one. I'll have to do some research on this...



To install ISCSI on AIX - from NetApp's site:

To install iSCSI on AIX 5.2 ML3 follow the steps below:

1. Verify the Maintenance level server is running:

# oslevel -r
5200-03

2. Verify the correct iSCSI FileSets have been installed:
# lslpp -l | grep -i iscsi
devices.common.IBM.iscsi.rte
5.2.0.30 COMMITTED Common iSCSI Files
devices.iscsi.disk.rte 5.2.0.30 COMMITTED iSCSI Disk Software
devices.iscsi.tape.rte 5.2.0.0 COMMITTED iSCSI Tape Software
devices.iscsi_sw.rte 5.2.0.1 COMMITTED iSCSI Software Device Driver
devices.iscsi_sw.rte 5.2.0.1 COMMITTED iSCSI Software Device Driver

3. Verify the iSCSI NodeName on the filer:

filer> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.33580086

4. Add the IP Address, default iSCSI port number, and iSCSI NodeName of the filer:

# tail -2 /etc/iscsi/targets
10.00.000.188 3260 iqn.1992-08.com.netapp:sn.33580086

5. Verify the iSCSI device is available in the kernel:

# lsdev -C | grep iscsi
iscsi0 Available iSCSI Protocol Device

6. Verify the iSCSI NodeName of the AIX server:

# lsattr -El iscsi0
disc_filename /etc/iscsi/targets Configuration file False
disc_policy file Discovery Policy True
initiator_name iqn.hostid.0a3cc1dc iSCSI Initiator Name True
max_targets 16 Maximum Targets Allowed True
num_cmd_elems 200 Maximum number of commands to queue to driver True

7. Modify the iSCSI NodeName of the AIX server:

NOTE: AIX’s default initiator nodename does not fully comply with RFC and filer nodename requirements.
A sample example is given below:
#lsattr -El iscsi0
nodename.iqn.aixhost1.hostid.0a3cac46
Change the default initiator nodename to include a date in the second field:

iqn.yyyy-mm.hostname.hostid.0a3cac46
Use the following to change the nodename:
chdev -l iscsi0 -a initiator_name=iqn.1996-04.aixhost1.hostid.0a3cac46
For this example it will be as shown below:
# chdev -l iscsi0 -a initiator_name=iqn.1992-08.com.ibm:aix-rtp.0a3cc1dc
iscsi0 changed

8. Verify the AIX server is logging into the filer:

filer> Wed Jun 9 14:40:57 EDT [iscsi.notice:notice]: ISCSI: iswta, New session from initiator iqn.1992-08.com.ibm:aix-rtp.0a3cc1dc at IP addr 10.00.00.00

9. Create an iSCSI igroup on the filer that contains the iSCSI NodeName of the AIX server:

filer> igroup create -i aix iqn.1992-08.com.ibm:aix-rtp.0a3cc1dc
where <iqn.1992-08.com.ibm:aix-rtp.0a3cc1dc> is the AIX initiator's nodename

10. Create some LUNs on the filer:

filer> lun create -s 1g /vol/test/aix.lun
filer> lun create -s 1g /vol/test/aix_5L.lun

11. Map the LUNs to the igroup for the AIX server:

filer > lun map /vol/test/aix.lun com.ibm:aix-rtp-iscsi
lun map: auto-assigned com.ibm:aix-rtp-iscsi=1

Verify that the lun are mapped

filer> lun show -g aix
/vol/test/aix.lun 1g (1073741824) (r/w, online, mapped)
/vol/test/aix_5L.lun 1g (1073741824) (r/w, online, mapped)

12. Verify that the block devices have not been created for you on the AIX server:

# lsdev -Cc disk
hdisk0 Available 10-60-00-9,0 16 Bit SCSI Disk Drive
hdisk1 Available 10-60-00-8,0 16 Bit LVD SCSI Disk Drive

13. Rescan the iSCSI subsystem for new devices:

# cfgmgr -l iscsi0

14. Verify that the block devices have been created for you on the AIX server:

# lsdev -Cc disk
hdisk0 Available 10-60-00-9,0 16 Bit SCSI Disk Drive
hdisk1 Available 10-60-00-8,0 16 Bit LVD SCSI Disk Drive
hdisk2 Available Other iSCSI Disk Drive
hdisk3 Available Other iSCSI Disk Drive


Regards,
Chuck

 

[cspilman]

BTW - even though the documentation shows this setup for AIX 5.2 ML3, I have setup and tested this with AIX5.3 ML3 and worked just fine.

The only issues I had with ISCSI were performance. I was told, though, that there are some things that could be tuned with ISCSI and the network to combat this but didn't really invest a lot of time into the resolution.

Regards,
Chuck

 

[cccompton]

Thank you Chuck for your detailed answer. I appreciate it.
Since I'm new to iSCSI, I hope you won't mind a couple of follow-up questions.

As I understand it, there a iSCSI Software Initiator which uses a standard Ethernet adapter and a hardware based iSCSI implementation using the iSCSI adapter. I believe the procedure above is for the iSCSI Software initiator.

I am guessing that setting up the Hardware initiator is similar except that instead of configuring "iscsi0" you configure the device "ics0". The principal steps being

(a) check for driver "devices.pci.1410d002" ( versus "devices.iscsi.disk.rte", etc. for software iSCSI)
(b) set iSCSI initiator node name
(c) create a /etc/iscsi/targetshw0 file (versus "targets")
(d) set IP, subnet mask, gateway for adapter
(e) set up the filer, same as you describe above
(f) discover LUNs using "cfgmgr -l ics0" (versus cfgmgr -l iscsi0")

I am "guessing" this because I can't find any detailed information and don't have a system to try it. Any comments?

----------------------------------------------------
New question:

We are trying to decide whether to use iSCSI at all. We have been told by our IBM pre-sales engineer that only iSCSI attach to NetApp applicance is supported for VIO server (versus, for example, NFS mounts).

My question is: is iSCSI straight forward and stable or is it a pain to get working and prone to problems? I'm hoping that by using an iSCSI adapter (and thereby offloading some of the processing load) and tuning network parameters performance can be made acceptable.

Feedback on iSCSI in general would be appreciated.

Thanks,
CC

 

[cspilman]

Yes, you are correct. The steps I listed are for the software ISCSI initiator setup. I have not setup the ISCSI hardware initiator so I my guess is as good as yours as to the proper setup and configuration.

I will agree with you that I believe that it should be the same as the setup above except for the steps that you mentioned but then again... I am just guessing to.

If you do try this, I would be interested in seeing the final steps to make it work.

As far as ISCSI being straight forward - yes, it was for me when setting up the ISCSI software initiator and then configuring it to work with our NetApp filer. However, I will state again, we did have problems with performance but they were minor and didn't have time to really sit down and investigate.

Sorry I couldn't give anymore information than what I did.

Regards,
Chuck