...making Linux just a little more fun!

<-- prev | next -->

# Roll your Desktop into a USB stick/CD

By Kapil Hari Paranjape

### Introduction

Here is a short course (with some excursions) to help you take your existing Desktop, roll it into an ISO pipe and smoke a USB stick or CD. This is not a task for the faint-hearted - there are no automated scripts in here!

Part of the reason for this hands-on approach is that each desktop system will have its own quirks, so writing scripts which handle the various conditions which might arise is painful. Secondly, there are numerous choices possible - you must mix your own. Finally, I must confess that I do not feel energetic enough to write the scripts at this point. Instead, you have this article!

So if you have spent a lot of time fine-tuning your configuration and want to waste some more time putting it onto a (re)movable drive - read on.

### Making a read-only root filesystem

Since the chosen medium for our ``live'' portable system is a CD or USB stick we do not want to write to it often. In fact, in the case of a CD-R we can only write to it once. It should be obvious, however, that a ``live'' system does need to write something if it is to be counted as among the living!

While many mechanisms have been suggested to handle this, we will (essentially) follow the system chosen by `bootcd`. To do this, we need to create a directory `/wraith`, an archive `/wraith.cpio.gz` and a script /etc/rcS.d/S01undead.sh.

The contents of the archive are rather system specific so you will need to choose its contents. However, if you are really, really impatient you can download the listing of the archive that I use and run the following:

     cd /
     cat wraith.lst | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gz

Did you get a lot of error messages? No? Are you really sure that your system is almost identical to mine? Yes? Then you can skip the next subsection.

### Choices, choices...

How does one find out what files need to be written to on a live system? One way would be to `find` all the files that have been written to on your current desktop. To do this first find out when the system was booted - a good measure of this is when the root filesystem was last mounted. So for example

     # Use your real root device in place of /dev/root.
     last_boot_time=$(dumpe2fs -h /dev/root | \
      sed -n -e's/Last mount time: * //p')

You may wish to use the log files or the `last` command instead. You only need to know the last boot time approximately; subtract a minute or so from it just to be on the safe side - unless you boot the system more often than that! Now create a file with that time stamp using the `touch` command:

     touch -d "$last_boot_time" /tmp/lastboot

You can now create the list of all files that were modified since that last boot (for simplicity we will only bother with the directories `/etc` and `/var`; you can add some other directories if you so desire):

     find /etc /var -newer /tmp/lastboot > /tmp/changed
     # And, just for fun...
     find /home -newer /tmp/lastboot > /tmp/home_changed

Have a look at these lists but don't delete them just yet. You should notice that there are three types of files that are written to on a running system.

- Files that are automatically written to but can be empty at the start like log files or the `mozilla` cache directories.
- Files that have information which we may want to change but rarely. Most configuration files fall into this category.
- Automatically generated databases of system information. Since our system is going to be ``static'' we actually do not want the system to update these databases. Thumb-rule: If the file is large in size it probably belongs here!

We will split up our list, `/tmp/changed`, according to this classification: `/tmp/write` will consist of those files (mostly directories) that are empty at boot time but get written to as the system runs; `/tmp/links` will consist of the files that will be quasi-static - we will keep a static version of these files at boot time but we might want to change them on a running system. We will include the third category of files in `/tmp/links` as well, since we will not treat them differently - but ultimately you may want to change this.

We first create a directory to hold the files that will be modifiable at run-time - say `/wraith`. Mount a RAM-based file system on it by `mount -t tmpfs tmpfs /wraith`. Big Fat Warning: This file system is ephemeral and will be lost when you halt the system. If you wish, you can use the directory as-is (without the `tmpfs` mount) during this subsection, but don't forget to clean up its contents once you have created the archive as explained below.

In `/wraith`, we will create the top-level directories like `etc`, `var`, `tmp` and so on that we will want to write to. In these directories we will create the files as per the classification above. First, we'll do the writable but empty files:

     cd /
     cat /tmp/write | cpio -pdum /wraith

We expect these files to be empty at start-up so we will ``zero'' them. Do this only to the files for which you don't want to keep the contents. For now I assume these are all the files in the list `/tmp/write`:

     for file in $(cat /tmp/write)
     do
      if [ -f $file ]
      then
       > /wraith/$file
      fi
     done

Of course, we also need an empty `tmp` directory:

     mkdir /wraith/tmp
     chmod 1777 /wraith/tmp

Next, we create the links.

     cd /
     for i in $(cat /tmp/links)
     do
      dir=$(dirname $i)
      top=$(echo $dir | cut -f2 -d'/')
      rest=$(echo $dir | cut -f3- -d'/')
      name=$(basename $i)
      mkdir -p /wraith/$dir
      ln -s /wraith/$top.ro/$rest/$name /wraith/$i
     done
     # As a safety measure to ensure that all configuration files
     # have been created
     mkdir -p /wraith/etc
     cd /etc
     for i in *
     do
      ln -s /wraith/etc.ro/$i /wraith/etc/$i
     done

This is more complicated and needs further explanation. The idea is to make the ``static'' versions of the files available under the `.ro` top-level directories. So, for example `/wraith/etc/hostname` will be a link to `/wraith/etc.ro/hostname`.

To see this at work create `etc.ro` and `var.ro` as sub-directories in `/wraith`. For each of these directories (say `etc`) we run a pair of commands like the following. (Warning: Be careful here. If you haven't created all the links in `/wraith/etc` as above you may crash your running system).

     mount --bind /etc /wraith/etc.ro
     mount --bind /wraith/etc /etc

After these mounts, the file `/etc/hostname` is a link to the original `hostname` file which is now available as `/wraith/etc.ro/hostname`. Since the left-half of this link is on the RAM disk we can perform replacement surgery on it:

     vi /etc/hostname.new
     mv /etc/hostname.new /etc/hostname

On the other hand, if you want to change a file in a sub-directory of `/etc`, it's a little more complicated:

     mkdir /etc/X11.new
     ln -s /ram/etc.ro/etc/X11/* /etc/X11.new
     mv /etc/X11.new /etc/X11

After this you can edit the files in `/etc/X11`. Yes, this is quite twisted but (once you get the hang of it) not difficult to manage - especially since we expect that we will edit these files only rarely. An alternate approach is to create the directory tree under `/etc` in its entirety only leaving links to the files.

You can use the above mounts to test your choices of `/tmp/links` and `/tmp/write`, but the real test will come later. For now, undo the above mounts by a pair of commands like:

     umount /etc
     umount /wraith/etc.ro

You can also remove the `.ro` directories if you like. Finally, we create an archive of this directory:

     cd /wraith
     find . -xdev | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gz

The `cpio` command will tell you how many 512-byte blocks you wrote. If the archive is really large (more than 1MB or so) then you probably need to re-do your choices.

### Bringing the wraith to life

We need a mechanism to bring the `/wraith` directory into operation at boot time. To do this, install a script like the following one to run early at boot time. For example you could install the script as `/etc/rcS.d/S01undead.sh`.

     #

     # undead.sh Mount and load up the /wraith directory for use
     #
     # Version: 0.3  01-Feb-2005
     #

     # If this has already been run then don't run it again.
     # We can't handle two wraiths!
     [ -f /wraith/live ] && exit 0

     # Create writable space
     mount -n -t tmpfs tmpfs /wraith

     # Create the directory structure
     cd /wraith
     gzip -dc /wraith.cpio.gz | cpio -idum
     cd /

     # Perform the cross mounts with bind
     # which is like a directory hard link.
     cd /wraith
     for i in *
     do
      mkdir $i.ro
      # We use mount with the -n
      # To avoid confusing the mtab
      mount -n --bind /$i /wraith/$i.ro
      mount -n --bind /wraith/$i /$i
     done
     cd /

    touch /wraith/live

    : exit

Finally, you edit `/etc/fstab` so that the root filesystem is mounted read-only at the next boot - just change `defaults` to read `ro,defaults` in the appropriate entry.

### Don't stop just yet

Reboot and that's it! You have a read-only root system... or almost. Actually, it is likely that you will find a number of places where you didn't create the links you need or created the wrong links. Don't worry. You can modify the `/wraith.cpio.gz` archive to your heart's content. Make the changes you need to the ``live'' `/wraith` directory. Now copy all the changes from `/wraith` into `/tmp/ghost`. The command

     cd /wraith
     find . -xdev | grep -E -v '^./((live)|([^/]*\.ro))' > /tmp/list

will generate the newer list of files. You can unpack the older archive and compare its list of files with `/tmp/list`.

     mkdir /tmp/ghost
     cd /tmp/ghost
     zcat /wraith.cpio.gz | cpio -idum
     find . -xdev > /tmp/oldlist
     wdiff -a /tmp/list /tmp/oldlist

Using the differences you can see what files you need to create in `/tmp/ghost` in order to match it up with the running `/wraith`. You can save your changes by something like

     cd /tmp/ghost
     find . -xdev | cpio -o -H crc | gzip -c -9 > /tmp/wraith.cpio.gz
     mv /tmp/wraith.cpio.gz /wraith.cpio.gz

The changes will become automatic at the next boot. Of course, once you write the filesystem to a CD, you will have no chance to revise it again!

### Undo

All this looks too complicated and life is too short? Just remove the script `/etc/rcS.d/S01undead.sh`, the archive `/wraith.cpio.gz` and the directory `/wraith`. You will have your system as pristine as before.

### Making the initrd

We want our system to ``run anywhere'' - in particular, we should be able to mount our root file system whether it resides on a CD or USB stick (or perhaps even hard disk). If we use a CD then on most systems this will be on the device `/dev/hdb` or `/dev/hdc`. The USB stick usually shows up as `/dev/sda` or `/dev/sdb`. It should be relatively simple to just create a kernel which supports IDE CD drives and USB block devices. When we boot such a kernel (with the correct `root=<device>` parameter) the system will start up as expected on 90% of all systems that one is likely to encounter. If this is OK with you then you don't need an initrd so skip the rest of this section and read the HOWTO on building the kernel with IDE CD and US support - don't forget support for the ISO 9660 (CD), ext2 and vfat (Win95) file systems.

What about the remaining 10%? That will take 90% of the work as usual. One possible solution could be to build all the drivers of all possible CD drives, USB readers and the like into the kernel. Unfortunately, automatically probing for some of these devices will occasionally cause other devices to choke-up. It also seems like a bit of a waste to take up such enormous amounts of kernel memory for unused drivers. The solution provided by our intrepid kernel hackers is the modules+initrd mechanism which allows you to write a script that chooses which drivers to load depending on the devices found.

### The steps

The boot loader (see the next section) will load the kernel and the initrd into memory. We will use a ``standard'' Debian kernel image which has essentially everything modularized (``essentially'' since we must have support for at least one file-system built into the kernel in order to load the init RAM disk - this could change if I understand `initramfs` better).

After the kernel has done its thing, it sets up the file-system with root on the initrd and executes `/linuxrc` but doesn't quite fully let go - when `/linuxrc` exits, the kernel executes `/sbin/init`. We follow Debian's choices when we visualize the boot process as follows:

linuxrc
  This script doesn't do much since we want the kernel to let go and execute `/sbin/init` (still on the initrd).
init
  The program `/sbin/init` on the initrd is a script that will run the following scripts:
  loadmodules
    This script loads the modules necessary to read the CD and/or USB stick.
  script
    This script will provide the subroutine `mount_root` to recognise and mount our chosen file system on `/mnt`.
  The final steps of `init` are
  - Mount the root file system on `/mnt` and `cd` to it.
  - Execute `pivot_root` which makes the current directory the root directory for the kernel and mounts the old root directory at `/initrd`. After this our ``real'' root file-system is mounted as root.
  - Execute `chroot .` to change the root device of the current process so that `/initrd` is free to be unmounted. We must do this so that the RAM disk is free to be unmounted which frees its space for use by other processes.
  - Finally execute `/sbin/init` on the real root file system. This is the ``real'' `init` program which will initialise the live system.

Debian has already provided the package initrd-tools which automates the building of the initrd. This already creates the `/linuxrc` and `/sbin/init` needed for the initrd. So we only need to provide the scripts `loadmodules` and `script`.

### Driver loading

Writing these scripts was one of the most complex steps for me as it deals with the aspect of Linux that I usually encounter the least - at least on a working system - booting! On the other hand, this is the job for which most installers and other forms of pre-install detection tools have been written. So we follow the ``teacher'' Tom Lehrer's dictum:

       Plagiarize,
       Let no one else's work evade your eyes,
       Remember why the good Lord made your eyes,
       So don't shade your eyes,
       But plagiarize, plagiarize, plagiarize...
       (Only be sure always to call it please, "research".)

       -- Tom Lehrer, "Lobachevsky"

There is a good IDE driver detection routine that is part of the standard Debian initrd. The Knoppix initrd gives us a safe order in which to load all the SCSI modules. The Linux-Live initrd has a list of the necessary USB modules to boot off a stick. So we put all these together to get routines which I call `loadmodules_ide`, `loadmodules_scsi` and `loadmodules_usb`. The `loadmodules` script on the initrd will then act as a dispatcher - it will choose which routine to run depending on what boot time parameters we give.

Still, we should do some work. So (plagiarising from the hotplug scripts) I also wrote a procedure `loadmodules_pci` that loads only those modules which correspond to devices in `/sys/bus/pci/devices` which match the data found in `/lib/modules/kernel-version/modules.pcimap`. This procedure makes use of the `sysfs` file system that was introduced with Linux 2.6.x but something similar may be possible using `/proc/bus/pci` in Linux version 2.4.x. The principle is that the kernel does provide a list of all the PCI devices that it found; for each such device it also provides some device information - the interface for this is the `sysfs` file system or (in 2.4.x) the `proc` file system. On the other hand, each module writer makes a list of all devices that the driver is known/expected to work for - the kernel build process writes these to `modules.pcimap`. By matching the two lists we should be able to load only those modules which have a matching device. This only works with PCI devices but most devices on PC's nowadays (including SCSI cards and the USB controller) are PCI devices.

Here is the script to loadmodules that resulted from the above deliberations. This scripts depends on a list of modules that are related to block devices.

### Mounting the root file-system

The second script we will use provides the routine to mount the root device. Again the `sysfs` file system provided by the 2.6.x Linux kernel comes to the rescue. Under `/sys/block` we find a list of all the block devices on the system. If the `root=` option is given to the kernel we can check whether this block device is available. Otherwise we check each available block device to find evidence that it is our root file system - by checking for the existence of the archive, directory and script that we created above.

### Using `mkinitrd`

The Debian initrd-tools package is a collection of scripts and so can be installed on any GNU/linux system (for example use the source package directly). The main script is `mkinitrd` which will create the standard Debian initrd. We will run this script and make some changes in order to create our ``special'' initrd. First off all create some directory say `/tmp/mkinitrd.confdir`. In this directory we will create the file `exe` containing the list of executables that we want in addition to the ``standard'' ones like `/bin/sh` - in our case we need `/bin/grep`. Next we create a list of all the additional files that we want to include; this is mainly the list of all modules that are in some way connected with the use of block devices; here is my list. Finally, we also need a configuration file. We are set to use `mkinitrd` with this directory as our configuration directory.

    mkinitrd -r "" -k -d /tmp/mkinitrd.confdir -o /dev/null

This will tell you the name of the working directory which will be something like `/tmp/mkinitrd.1234`. Now you need to edit the `/tmp/mkinitrd.1234/initrd/linuxrc.conf` file to reflect the various file systems that you may use for your root file system. Finally, you copy the scripts you created above and generate the initrd with `mkcramfs`.

    dir=/tmp/mkinitrd.1234
    rm $dir/initrd/scripts/*
    cp allmod.list $dir/initrd/etc
    cp loadmodules $dir/initrd
    cp script $dir/initrd
    chmod +x $dir/initrd/loadmodules
    chmod +x $dir/initrd/script
    mkcramfs $dir/initrd initrd.img

If you build a kernel with `ext2` filesystem support instead of `cramfs`, then you need to create an `ext2` filesystem image based on the `/tmp/mkinitrd.1234` directory instead.

### Putting it all together

We now combine the ideas of the previous two sections. I assume that you have managed to make your root filesystem boot in a ``read-only'' mode and that you are currently running in that mode. I also assume that you have created an initrd that can boot on ``any'' machine.

I know that the latter requirement is hard to check given that you have access to only one machine at a time. Moreover, it is difficult to find friends who will agree if you say ``I have on this floppy an initrd and kernel that I would like to test on your system''; those few will not remain friends if your kernel+initrd manages to fry their system.

### Selecting the boot loader

In order to boot off a CD or USB stick we need some software that can do that. The nominees are `isolinux`, `loadlin` and `grub`... and the winner, in this case, is... `grub`.

The main file for booting using `grub` is called `stage2` or `stage2_eltorito` in the case of booting CD's. When these files are properly installed (see below how this is done for CD's), they are loaded and run by the booting machine. They look for a configuration file `/boot/grub/menu.lst`. We use a `menu.lst` file that looks like:

    default  0

     timeout  5

     # Pretty colours
    color cyan/blue white/blue

     title  Debian GNU/Linux with myinitrd
    root  (cd)
    kernel  /boot/vmlinuz-2.6.8-2-686 root=auto ro quiet vga=791
    initrd  /boot/initrd.img
    boot

Other than the kernel and the initrd, we need `stage2` and `menu.lst` in order to complete the list of steps given at the beginning.

### Making the CD

First you need a ``pristine'' copy of the root file system. If you used the `bind` mounts procedure to make the root file-system read-only, then you can just do

     mkdir /tmp/pristine
     mount --bind / /tmp/pristine

You then make a compressed tree of this file system:

     mkzftree -x /tmp/pristine /hugeroom

where `/hugeroom` is some place with a lot of disk space. Remove the directories `/hugeroom/lost+found` and `/hugeroom/boot` from under this directory. Create an empty `/hugeroom/boot` directory to which we copy the kernel image and initrd. Into the `/hugeroom/boot/grub` directory goes the file `stage2_eltorito` along with `menu.lst`. These files will not be compressed.

We now create the CD image:

     mkisofs -R -J -z -hide-rr-moved -cache-inodes \
      -b boot/grub/stage2_eltorito -b boot/boot.cat \
      -boot-info-table -boot-load-size 32 \
      -no-emul-boot -o mylivecd.iso /hugeroom

Then we blank a CD (if necessary) and write our image to it. For a USB stick, we just create a partition and dump the entire image to this partition using `dd`. Since I do not have a system that can boot off a USB, I can only check the floppy based boot for such a system. Perhaps one of the readers can enlighten me on how this is to be handled for USB-booting BIOSes.

### What else?

You'll probably want to add a writable `/home` directory to your system. You need to further customise `wraith.cpio.gz` for that. Another thing that you probably want to do is to fix the `/etc/fstab` file that goes onto the CD. Other config files may also need to be customised; `/etc/X11/XF86config-4` comes to mind - for this to work ``anywhere'' it is best to use the `vesa` driver. Similarly, use `dhcp` to configure ethernet rather than a hard-coded IP address in `/etc/network/interfaces`. On most systems there is a hard disk and it is shame not to use it. You can set-up a swap partition after you boot from the CD - be careful not to trash the host machine though!

### Afterword

Today one can find a number of GNU/Linux systems that work off Live CD's. There is Knoppix - and then there are its Klones. There is LNX-BBC, tomsrtbt, LTSP and even one called Puppy! There are the CD-based installers for the common distributions. But, I am still not satisfied. Each of these make choices that I am not comfortable with. They choose KDE, Gnome or `fluxbox`, when what I want is `fvwm`; or they choose `xmms` when what I want is `alsaplayer` (in daemon mode)... and so on.

What's wrong with Sunil's excellent article then? - just take a minimal Knoppix-like DSL and re-master it. I would object that Knoppix puts everything in a `cloop` image which makes it difficult to read the ``real'' contents of the CD on a generic system; further this also makes it difficult to master and/or re-master.

There are other approaches like that taken by Gibraltar, `bootcd` or `dfsbuild` or `linux-live`.

The first two keep the files in a compressed ISO file-system. That makes it usable ``anywhere''. I did try these but for one reason or another they didn't work for me. For example they required the installation of additional packages on my desktop.

Ultimately, it comes down to this: I'm a terribly nit-picky kind of person, and I have spent a lot of time fine-tuning my system and no one is allowed to dictate what packages I must install and how they must be configured.

[ I like this Kapil guy, and the way he thinks. :) -- Ben ]

I enjoy tinkering with such things, and so I must have a system that I understand fully. People also mentioned additional kernel features in late 2.4.x and early 2.6.x that simplify the building of a live CD. Finally, isn't it fun to ``roll your own''?

> This document was translated from the LaTeX Source by HEVEA .

Kapil Hari Paranjape has been a ``hack''-er since his punch-card days. Specifically, this means that he has never written a ``real'' program. He has merely tinkered with programs written by others. After playing with Minix in 1990-91 he thought of writing his first program---a ``genuine'' *nix kernel for the x86 class of machines. Luckily for him a certain L. Torvalds got there first---thereby saving him the trouble (once again) of actually writing code. In eternal gratitude he has spent a lot of time tinkering with and promoting Linux and GNU since those days---much to the dismay of many around him who think he should concentrate on mathematical research---which is his paying job. The interplay between actual running programs, what can be computed in principle and what can be shown to exist continues to fascinate him.

Copyright © 2005, Kapil Hari Paranjape. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 113 of Linux Gazette, April 2005

<-- prev | next -->