iSCSI and Anaconda¶

Introduction¶

iSCSI device is a SCSI device connected to your computer via a TCP/IP network. The communication can be handled either in hardware or in software, or as a hybrid — part software, part hardware.

The terminology:

• ‘initiator’, the client in the iscsi connection. The computer we are running Anaconda on is typically an initiator.
• ‘target’, the storage device behind the Network. This is where the data is physically stored and read from. You can turn any Fedora/RHEL machine to a target (or several) via scsi-target-utils.
• ‘HBA’ or Host Bus Adapter. A device (PCI card typically) you connect to a computer. It acts as a NIC and if you configure it properly it transparently connects to the target when started and all you can see is a block device on your system.
• ‘software initiator’ is what you end up with if you emulate most of what HBA is doing and just use a regular NIC for the iscsi communication. The modern Linux kernel has a software initiator. To use it, you need the Open-ISCSI software stack [1, 2] installed. It is known as iscsi-initiator-utils in Fedora/RHEL.
• ‘partial offload card’. Similar to HBA but needs some support from kernel and iscsi-initiator-utils. The least pleasant to work with, particularly because there is no standardized amount of the manual setting that needs to be done (some connect to the target just like HBAs, some need you to bring their NIC part up manually etc.). Partial offload cards exist to get better performing I/O with less processor load than with software initiator.
• ‘iBFT’ as in ‘Iscsi Boot Firmware Table’. A table in the card’s bios that contains its network and target settings. This allows the card to configure itself, connect to a target and boot from it before any operating system or a bootloader has the chance. We can also read this information from /sys/firmware/ibft after the system starts and then use it to bring the card up (again) in Linux.
• ‘CHAP’ is the authentication used for iSCSI connections. The authentication can happen during target discovery or target login or both. It can happen in both directions too: the initiator authenticates itself to the target and the target is sometimes required to authenticate itself to the initiator.

What is expected from Anaconda¶

We are expected to:

• use an HBA like an ordinary disk. It is usually smart enough to bring itself up during boot, connect to the target and just act as an ordinary disk.
• allow creating new software initiator connections in the UI, both IPv4 and IPv6.
• facilitate bringing up iBFT connections for partial offload cards.
• install the root and/or /boot filesystems on any iSCSI initiator known to us
• remember to install dracut-network if we are booting from an iSCSI initiator that requires iscsi-initiator-utils in the ramdisk (most of them do)
• boot from an iSCSI initiator using dracut, this requires generating an appropriate set of kernel boot arguments for it [3].

How Anaconda handles iscsi¶

iSCSI comes into play several times while Anaconda does its thing:

In loader, when deciding what NIC we should setup, we check if we have iBFT information from one of the cards. If we do we set that card up with what we found in the table, it usually boils down to an IPv4 static or IPv4 DHCP-obtained address. [4][5]

Next, after the main UI startup during filtering (or storage scan, whatever comes first) we startup the iscsi support code in Anaconda [6]. This currently involves: - manually modprobing related kernel modules - starting the iscsiuio daemon (required by some partial offload cards) - most importantly, starting the iscsid daemon

All iBFT connections are brought up next by looking at the cards’ iBFT data, if any. The filtering screen has a feature to add advanced storage devices, including iSCSI. Both connection types are handled by libiscsi (see below). The brought up iSCSI devices appear as /dev/sdX and are treated as ordinary block devices.

When DeviceTree scans all the block devices it uses the udev data (particularly the ID_BUS and ID_PATH keys) to decide if the device is an iscsi disk. If it is, it is represented with an iScsiDiskDevice class instance. This helps Anaconda remember that:

• we need to install dracut-network so the generated dracut image is able to bring up the underlying NIC and establish the iscsi connection.
• if we are booting from the device we need to pass dracut a proper set of arguments that will allow it to do so.

Libiscsi¶

How are iSCSI targets found and logged into? Originally Anaconda was just running iscsiadm as an external program through execWithRedirect(). This ultimately proved awkward especially due to the difficulties of handling the CHAP passphrases this way. That is why Hans de Goede <hdegoede@redhat.com>, the previous maintainer of the Anaconda iscsi subsystem decided to write a better interface and created libiscsi (do not confuse this with the libiscsi.c in kernel). Currently libiscsi lives as a couple of patches in the RHEL6 iscsi-initiator-utils CVS (and in Fedora package git, in somewhat outdated version). Since Anaconda is libiscsi’s only client at the moment it is maintained by the Anaconda team.

The promise of libiscsi is to provide a simple C/Python API to handle iSCSI connections while being somewhat stable and independent of the changes in the underlying initiator-utils (while otherwise being tied to it on the implementation level).

And at the moment libiscsi does just that. It has a set of functions to discover and login to targets software targets. It supports making connections through partial offload interfaces, but the only discovery method supported at this moment is through firmware (iBFT). Its public data structures are independent of iscsi-initiator-utils. And there is some python boilerplate that wraps the core functions so we can easily call those from Anaconda.

To start nontrivial hacking on libiscsi prepare to spend some time familiarizing yourself with the iscsi-initiator-utils internals (it is complex but quite nice).

Debugging iSCSI bugs¶

There is some information in anaconda.log and storage.log but libiscsi itself is quite bad at logging. Most times useful information can be found by sshing onto the machine and inspecting the output of different iscsiadm commands [2][7], especially querying the existing sessions and known interfaces.

If for some reason the DeviceTree fails at recognizing iscsi devices as such, ‘udevadm info –exportdb’ is of interest.

The booting problems are either due to incorrectly generated dracut boot arguments or they are simply dracut bugs.

Note that many of the iscsi adapters are installed in different Red Hat machines and so the issues can often be reproduced and debugged.

Future of iSCSI in Anaconda¶

• extend libiscsi to allow initializing arbitrary connections from a partial offload card. Implement the Anaconda UI to utilize this. Difficulty hard.
• extend libiscsi with device binding support. Difficulty hard.
• work with iscsi-initiator-utils maintainer to get libiscsi.c upstream and then to rawhide Fedora. Then the partial offload patches in the RHEL6 Anaconda can be migrated there too and partial offload can be tested. This is something that needs to be done before RHEL7. Difficulty medium.
• improve libiscsi’s logging capabilities. Difficulty easy.