# Driver Update Disks Developer Documentation _v1.0, Mon Feb 15 2016_ This document describes implementation details of how Driver Update Disks (usually abbreviated DUD or DD) are loaded and used by the installer, as a reference for developers working on the code. **_NOTE: This is not a specification. Filenames, paths, and procedures may change without warning. Avoid contact with eyes and skin and avoid inhaling fumes. Keep out of reach of children and customers. Any resemblance to real code, living or dead is purely coincidental._** Most of the important code is in `driver_updates.py`, which uses the utilities from `utils/dd` to actually handle driver RPMs. There's also support code sprinkled through this directory and in the installer itself, which will be described below. But first... # A Quick Overview ## What is a DUD? A DUD (Driver Update Disk) is a disk (or disk image - typically a `.iso` file) that contains some _metadata_ and one or more _repos_ which contain _driver RPMs_ and/or _installer enhancement RPMs_. ## What are they for? These are used to add kernel modules and/or userspace programs to the installer *before* the installer's second stage is loaded. This is most commonly needed when trying to install onto a system whose disk (or network) controllers are not supported by the existing installer. ## How does it work? Here's a short summary; please see below for important technical details of each part of the process. 1. **Boot argument**: `inst.dd=` specifies where to look for driver disks. 2. **Automatic DUD handling**: If a disk partition labeled `OEMDRV` is found during startup (before `udev` settles) it's assumed to be a driver disk. 3. **Interactive DUD selection**: The user may pass the argument `inst.dd` by itself to request interactive driver disk selection via a text menu. The user can choose a device to examine, pick an `.iso` image to load from (if multiple images are present), and choose individual drivers to load. 4. **Driver RPM selection**: `dd_list` is used to list the contents of a DUD repo; it will only list RPMs that have one of these headers, matching the current kernel/installer version: * `Provides: kernel-modules ` * `Provides: installer-enhancement ` 5. **Driver extraction**: For all matching RPMs, `dd_extract` extracts kernel modules and firmware (if the `kernel-modules` header was present) and binaries and libraries (if `installer-enhancement` was present) into the initrd and installer environment. 6. **Driver loading**: After processing each DUD, `driver_updates.py` runs `depmod` and tries to manually `modprobe` all extracted drivers. 7. **Repo copying and package listing**: `driver_updates.py` saves a list of the extracted package names and a copy of each DUD repo used into the installer environment. When `anaconda` runs, it enables any DUD repos it finds and adds all the named packages to the install transaction. # Overall design notes Here are a few things to keep in mind about the overall design of the DUD code. 1. **The real action happens in the dracut initqueue** We need both `udev` and `dracut` to be processing events so that other dependent events are handled as usual. It's tempting to think that we should just handle DUDs *before* the initqueue mainloop so that the devices will magically already be present when we start the initqueue, as if we had started with them loaded. Except then you need to start running `udevadm trigger` and `udevadm settle` yourself to make the DUD devices appear, and then handle them manually, and now you've gone and re-written the dracut mainloop. Too clever is stupid. Just use initqueue. *Any long-running action that's going to be performed inside one of the dracut scriptlets should be using `/sbin/initqueue` to add items to the initqueue instead of executing the code itself in-place.* (See, for example, the way `driver-updates-genrules.sh` handles DUDs that are already present inside the initramfs.) 2. **Let `dracut` handle devices and downloads** `dracut`'s main purpose in life is to bring up and initialize the devices needed to start the system (or, in this case, the installer image). Dracut's far better at handling disks and image mounting than Python is, and in general we want upstream dracut to be responsible for handling as much as possible of the bootup process. (That's kind of its job, after all.) *Anything that directly handles probing devices, downloading images, calling udev, etc. should be done in dracut scriptlets, not `driver_updates.py`.* (The current invocation of `udevadm` that's in there should be moved to a dracut scriptlet as soon as we can get away with changing it.) 3. **Let `dd_list`/`dd_extract` handle driver RPMs** Neither `dracut` nor `driver_updates.py` should be messing with RPM headers, unpacking individual RPMs, etc. *Use `dd_list` to list the driver RPMs that are available in a repo, and use `dd_extract` to extract their contents. Don't mess with RPMs directly.* 4. **Use as little Python as possible in initrd** If you know Python better than dracut or C, it might be tempting to solve problems by hacking on the Python parts of the DUD code. This leads to redundant code - and more code for us to maintain. `driver_updates.py`'s purpose is basically just to mount disks or disk images and find valid DUD repos therein (either automatically or interactively), and to save needed data to places where the installer can find it later. *Any other tasks should be handled in dracut scriptlets, the C utilities, or the main installer program.* (If you must add Python code, avoid pulling in new libraries. It makes building images much more fragile and bloats the initramfs.) # Boot arguments The `inst.dd` (or `dd`) boot argument is used to specify the location of a DUD to use, or to start interactive mode. It has three forms: 1. Network URLs * example: `inst.dd=http://host.fake.domain/path/to/dd.iso` * Target should be a `.iso` (or other disk image) or driver `.rpm` * Target will be downloaded and then handled appropriately 2. Block devices * examples: `inst.dd=cdrom:/dev/cdrom`, `inst.dd=hd:LABEL=DRIVERZ` * The `hd:` or `cdrom:` prefix is stripped * `TAG=VALUE` pairs are looked up using `blkid` 3. Interactive mode * `inst.dd` (or `dd`) * Brings up text-based menu: 1. Choose from a list of disk devices 2. (if device contains multiple `.iso` images) Choose an `.iso` 3. Choose from a list of drivers found in the DUD If one (or more) of these are specified, dracut will wait until all specified `inst.dd` items have been handled before starting the main installer image. # Automatic DUD handling (`OEMDRV`) If no `inst.dd` boot argument is specified, dracut watches for a readable disk partition labeled `OEMDRV`. If one appears before dracut finishes waiting for required devices, it is automatically handled like a normal DUD. ## NOTE: device autodetection limitations Once the kernel's device probing finishes, `udev` waits for and processes responses from devices. Once every response has been handled and no more new responses arrive for a short period of time (no less than 500ms), dracut assumes all devices have been found and declares the system "settled". If there's no response from any `OEMDRV` device by then, the installer starts normally. _If the `OEMDRV` device is present but too slow to be autodetected, the user can boot with an argument like `inst.dd=hd:LABEL=OEMDRV` to indicate that dracut should expect an `OEMDRV` device and not start the installer until it appears._ # DUD filesystem layout A driver updates disk may contain one or more driver repos, which are usually laid out as follows: / |rhdd3 - DD marker, contains the DD's description string /rpms | /i686 - contains RPMs for this arch and acts as package repo | /x86_64 | /ppc64 | /... - any other architecture the DD provides drivers for In other words, a DUD is a directory containing: 1. a file named `rhdd3` (which contains the description for that DUD), and 2. a directory named `rpms/`, under which there is at least one directory which matches the output of `uname -m`. Under there are RPMs that contain the special DUD headers (see the next section for more info). By convention, the "repos" under `rpms/` usually have yum metadata, but this is not required or enforced by the tools. If there's no metadata present, anaconda will run `createrepo` (or whatever the backend requires to make that directory usable). # Driver Updates RPM Headers When `dd_list` examines a driver disk, it checks the `Provides:` headers of each RPM to see if it provides either `kernel-modules` or `installer-enhancement`. RPMs that contain kernel modules or firmware must provide `kernel-modules`, and RPMs that contain libraries or binaries must provide `installer-enhancement`. ## RPM header version matching Usually the packages will have a header like one of these: Provides: kernel-modules >= 3.6.9 Provides: installer-enhancement >= 19 The given version comparison expression is compared against either the running kernel or the installer version; if the kernel or installer matches the expression, then the RPM is considered valid for use inside this installer environment, and it will be listed by `dd_list`. Otherwise, `dd_list` (and, therefore, the rest of the DUD code) will ignore that RPM. Note that RPM operations are handled entirely by `dd_list` and `dd_extract`. ### `installer-enhancement` versioning The DUD code does _not_ use the actual running installer version; instead it hardcodes the value `19`. I think (at some point) there _were_ DUDs in the wild that required this fact, and so it's just been left as-is. If the Anaconda Add-on API changes, this version should *definitely* be changed. # `dracut` component details This section describes all the various bits of driver update support code that live in `dracut` and how they get called or call each other. ## cmdline: `parse-anaconda-dd.sh` If any `inst.dd=...` (or `dd=...`) arguments are found, this scriptlet parses their values and writes appropriate values to `/tmp/dd_disk`, `/tmp/dd_net`, or `/tmp/dd_interactive`, as follows: 1. Values that start with `http:`, `https:`, `ftp:`, or `nfs:` are assumed to be network URLs and get written to `/tmp/dd_net`. 2. Values of the form `hd:`, `cdrom:`, `file:`, or `path:` will have the `` or `` part written to `/tmp/dd_disk`. 3. Any other value is assumed to be a disk device, and therefore also gets written to `/tmp/dd_disk`. 4. If a plain `dd` or `inst.dd` boot argument (without a value) is found, the word "menu" is written to `/tmp/dd_interactive`. Finally, the contents of all three files are combined to create `/tmp/dd_todo`. ## pre-trigger: `driver-updates-genrules.sh` This sets up the udev rules and dracut scriptlets that will handle any DUDs that appear. Specifically: 1. If `/tmp/dd_todo` exists, call `anaconda-lib.sh:wait_for_dd()` so that dracut will not start the installer until `/tmp/dd.done` exists. 2. If `/tmp/dd_interactive` exists, add an initqueue item to start `driver-updates@.service` once the initqueue starts. 3. If `/tmp/dd_interactive` does *not* exist, add a udev rule to automatically handle any `OEMDRV` devices that appear. 4. For each item specified in `/tmp/dd_disk`, either: * create a udev rule that will execute `driver-updates --disk` when that disk appears, or * if the item is a DUD image that already exists in the initrd: add an initqueue item to run `driver-updates --disk` once the initqueue starts. 5. Set up a `initqueue/finished` scriptlet to force dracut to stay in the initqueue until `initqueue/settled` runs at least once. ## initqueue/online: `fetch-driver-net.sh` This script runs every time a network device comes online and tries to handle each image URL listed in in `/tmp/dd_net`, as follows: 1. If the URL is listed in `/tmp/dd_net.done`, it is skipped 2. The URL is fetched using dracut's `fetch_url` function 3. If the image is successfully downloaded: * append URL to `/tmp/dd_net.done` * run `driver-updates --net URL IMAGE` 4. Otherwise: if the fetch failed, warn the user and exit ## `parse-kickstart` When `parse-kickstart` handles a `driverdisk` command it emits the appropriate `inst.dd=` argument, which will then get parsed and handled as usual when the `run_kickstart` function does its thing (see `anaconda-lib.sh` for details on how `run_kickstart` works). There's two valid forms for the `driverdisk` command - here's how those get handled by `parse-kickstart`: | kickstart command | `parse-kickstart` output |--------------------------------|-------------------------- | `driverdisk ` | `inst.dd=hd:` | `driverdisk --source=` | `inst.dd=` Check the [kickstart documentation] for more info. [kickstart documentation]: https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst#driverdisk ## initqueue: `driver-updates@.service` This service launches the interactive driver update menu. It handles hiding the `plymouth` splash screen, quieting kernel messages, and connecting the menu to the tty so that the user can select drivers. This gets started from the initqueue by `driver-updates-genrules.sh`, if interactive mode was requested. ## initqueue: `driver_updates.py` Handles a single DUD request: extract and load drivers, save the DUD repo(s) for later use, and keep track of which requests have been handled. (Also handles the interactive mode - more on that below.) **NOTE:** inside the initrd environment this file is named `/bin/driver-updates`. (See `module-setup.sh` if you want to know more about what files get put where inside the initrd.) ### Invocation For disk devices, `driver-updates-genrules.sh` sets up a udev rule that will run `/sbin/initqueue` to add an initqueue job which will run `driver-updates`. For initrd-embedded images, `driver-updates-genrules.sh` skips udev and runs `/sbin/initqueue` itself to add the job to the initqueue. For network images, `fetch-driver-net.sh` (in the `initqueue/online` hook) runs `driver-updates` itself. ### Arguments * `driver-updates --disk PART DEVNODE` * `driver-updates --net URL LOCALFILE` In both cases, the first argument is the string provided by the user - either the partition specifier (like `LABEL=DRIVERS`) or the URL. This string will also be in `/tmp/dd_net` or `/tmp/dd_disk`, and `/tmp/dd_todo`. The second argument is the actual image or device node to be mounted and processed. Dracut is responsible for downloading images or finding the disk device and passing that to `driver-updates` here. In the case of `inst.dd=file:/dd.img`, both items will be `/dd.img`. ### Driver Updates Disk Handling Here, roughly, is how `driver-updates` handles a DUD: 1. Mount the disk (the `DEVNODE` or `LOCALFILE` argument above) 2. For all valid `rhdd3` repos found, list the RPMs using `dd_list` * A valid repo is a directory containing a file named `rhdd3` and a subdirectory named `rpms/$ARCH`. (See [DUD filesystem layout](#dud-filesystem-layout), above) 3. For all matching RPMs found, `dd_extract` the RPM into `/updates` * `/updates` is overlaid onto the installer image when we switch there 4. If the package contains drivers, write the name of the package to `/run/install/dd_packages` 5. If the current repo had any driver RPMs, copy it to `/run/install/DD-1` (`DD-2`, `DD-3`, etc.) 6. Load all the drivers that were extracted: 1. Copy the drivers and firmware to the `updates/` dirs: * `/lib/modules/$(uname -r)/updates/` * `/lib/firmware/updates/` 2. Run `depmod -a` and then `modprobe -a ` 4. Append the `PART` or `URL` argument string to `/tmp/dd_finished` 5. If every item in `/tmp/dd_todo` is now also in `/tmp/dd_finished`, create `/tmp/dd.done` so dracut knows it can exit the initqueue. ### Bash helper scripts called from driver_updates.py * anaconda-ifdown * This script sets the interface down and removes all flags in dracut for future re-setting. This is useful for replacing existing network drivers. * find-net-intfs-by-driver * Find all network interfaces which depend on the given driver (command line argument), then return a list of the network interfaces. ## pre-pivot: `anaconda-depmod.sh` If any drivers were installed or downloaded, run depmod on `$NEWROOT` so we can load the drivers later, if needed. (To determine whether drivers were installed it simply checks to see if `/run/install/DD-1` exists.) # Testing Testing the DUD code is a little tricky, since it's pretty hardware dependent. Here are some tools and resources that should help if you're trying to test the DUD code. ## Integration tests & unit tests There are unit tests for `driver_updates.py` in `tests/dracut_tests/test_driver_updates.py`. Please do add test cases for any bugs that are found or new methods that get added. There are also some tests for `dd_list` and `dd_extract`, in `tests/dd_tests/dd_test.py`. These tests ensure that the utilities behave the way that `driver_updates.py` expects them to. (These tests run as part of the normal suite of tests that run during `make check`.) Finally, the [kickstart-tests] repo has at least one functional/integration test for the `driverdisk` command. It also contains a helper program, `mkdud.py`, which can be used to generate (fake) DUD images for test purposes. [kickstart-tests]: https://github.com/rhinstaller/kickstart-tests ## Real-world DUD images Here's a Red Hat Knowledgebase article that links to some Red Hat-provided images: https://access.redhat.com/articles/64322 (Sadly, you have to log in to download them) # References Old driverdisc docs from the RHEL7 [anaconda source]: * [docs/driverdisc.rst]: describes the driver disc format and motivations for its design. * [dracut/README-dd]: developer documentation on the dd code in dracut. [anaconda source]: https://github.com/rhinstaller/anaconda [docs/driverdisc.rst]: https://github.com/rhinstaller/anaconda/blob/rhel7-branch/docs/driverdisc.rst [dracut/README-dd]: https://github.com/rhinstaller/anaconda/blob/rhel7-branch/dracut/README-dd