Or, How To Get This Damn USB Flash Drive To Boot.

These notes are ok, but at least the author is trying to do the same thing I am trying to do which is unravel the mystical processes that take place (or fail to) inside of utility programs that try to automagically do this job.

An interesting reference is the man page for makebootfat which says this.

The BIOS USB boot support is generally differentiated in three
categories: USB-HDD, USB-FDD and USB-ZIP.

The USB-HDD (Hard Disk Drive) standard is the preferred choice and it
requires the presence of a partition table in the first sector of the
disk. You can create this type of disk using the -m option.

The USB-FDD (Floppy Disk Drive) standard requires the presence of a
filesystem starting from the first sector of the disk without a
partition table. You can create this type of disk without using the -m
option.

The USB-ZIP (ZIP Drive) standard requires the presence of a device
with a very specific geometry. Specifically, it requires a geometry
with 32 sectors and 64 heads. It also requires the presence of a
partition table with only a bootable partition in the fourth entry.
You can create this type of disk using the -m and -Z option.

Generally these standards are incompatible, but using the -m, -F and
-Z options you can create a disk compatible with all of them.

Partitioning

It is an open debate how or even if one should partition a USB drive. Ultimately I’d like to put the OS on a 10GB partition or so and have another partition with a sane (ext.*) file system to use with data. I have accomplished this with sysresc so I know it’s possible.

Ideally I could partition with no VFAT or FAT32 but I can’t figure out how at this time.

Device Inventory

It is super important to make sure you’re dealing with the right device. Plug the USB in and remove it a few times while keep in an eye on how that changes the results of lsblk if you need to. When you’re pretty sure you know exactly which drive designation the kernel is using to refer to your target drive, find out its raw max storage with something like this.

:-> [crow][~]$ lsblk -b /dev/sdb
NAME MAJ:MIN RM       SIZE RO TYPE MOUNTPOINT
sdb    8:16   1 8103395328  0 disk

This drive, /dev/sdb, is reporting exactly 8103395328 bytes.

CHS

Strangely the ancient legacy technology of Cylinders, Heads, and Sectors (CHS) pokes its wizened head from out of the grave.

Use old-fashioned fdisk to find some things out.

:-> [crow][~]$ sudo fdisk -l /dev/sdb

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdb: 8103 MB, 8103395328 bytes
255 heads, 63 sectors/track, 985 cylinders, total 15826944 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1    15826943     7913471+  ee  GPT

Even though fdisk doesn’t really work on this kind of partition table, it does tell us useful things about the (imaginary) disk structure.

Multiply the following things together.

  • 8103395328 bytes / drive

  • 1 sector / 512 bytes

  • 1 drive / 255 heads

  • 1 cylinder / 63 sectors

Canceling units gives 985.181699346 cylinders/head or simply cylinders. If you’re writing your own file system layout you might be able to use that .181699346 cylinder but in the real world we’ll have to just call this drive, as fdisk does, 985 cylinders.

In Linux, to find out how many bytes each block is on a block device use this command.

sudo blockdev --getbsz /dev/sdb

I think this value will very typically be 4096.

A Full Script

From the same reference, here’s my interpretation of the process.

flashinit
#!/bin/bash
function usage {
cat <<EOF
Must be run as root.
Usage:
   flashinit <partition>
Example:
   flashinit /dev/sdb1
EOF
}

if [ "${UID}" -ne "0" ]; then echo "UID=${UID}" ; usage ; exit; fi

# Set the drive and specific partition info.
PARTITION=$1
if [ -z "$1" ]; then usage ; exit ; fi

PARTITION=${PARTITION?}
FLASH=${PARTITION:: -1}
echo "Flash drive is $FLASH"
lsblk ${FLASH}
echo "If this is incorrect, press something other than enter now!"
read OK
if [ -n "$OK" ]; then echo "Aborting then... Nothing done." ; exit ; fi

# Make sure it's not mounted.
if umount ${PARTITION} 2>/dev/null
then
    echo "${PARTITION} was mounted, now unmounted..."
fi

# Zero out the MBR (first 1024 bytes).
echo "Zeroing the MBR..."
dd if=/dev/zero of=$FLASH bs=1024 count=1

#calculate sectors, cylinders
echo "Calculating reported drive geometry..."
SIZE=`fdisk -l $FLASH | grep $FLASH | awk '{print $5}'`
echo "Disk SIZE: ${SIZE} bytes"
CYLINDERS=`echo $SIZE/255/63/512 | bc`
echo "Cylinders: ${CYLINDERS}"
SECTORS=`echo $CYLINDERS*255*63 | bc`
echo "Sectors: ${SECTORS}"
SIZESECTORS=`echo $SECTORS-63 | bc`
echo "Size in sectors: ${SIZESECTORS}"

#create partition table and partition
echo "Creating partition table..."
sfdisk -uS -f -C${CYLINDERS} -H255 -S63 ${FLASH} << EOF
/dev/sdb1 : start= 63, size= ${SIZESECTORS}, Id= c, bootable
/dev/sdb2 : start= 0, size= 0, Id= 0
/dev/sdb3 : start= 0, size= 0, Id= 0
/dev/sdb4 : start= 0, size= 0, Id= 0
EOF

#format partition
echo "Formatting partition..."
mkfs.vfat -F 32 -n VFATUSB ${PARTITION}
sync
partprobe -s ${FLASH}

#install syslinux
echo "Installing syslinux..."
./syslinux -i $PARTITION

# Needs `apt-get install makebootfat`
MBRFILE=/usr/lib/makebootfat/mbrfat.bin
#write syslinux MBR
echo "Writing ${MBRFILE} to MBR of ${FLASH}..."
dd if=${MBRFILE} of=${FLASH} bs=446 count=1

Troubleshooting

When the partitions and the MBR are prepared but nothing else you might get an error (like my laptop) which says this.

HDD EBIOS
This is not a bootable disk. Please insert a bootable floppy and press
any key to try again.

This actually represents a level of success since that first line is the MBR successfully being hit by the BIOS and then looking for the actual boot loader (which is not present).

Ext Approaches

Syslinux has another derivative called extlinux which boots directly to ext2 file systems. All of these operate outside of normal distro support so they run pretty independently. This means if it’s not easily available as a package, just install it from source. Here is extlinux’s web page.

And here is a link to a typical download location. In some cases I had to install from source. It’s a well behaved tar file that compiles with a simple make. On CentOS7 I needed to install packages nasm and libuuid-devel.

It’s RPM is syslinux-extlinux. For Gentoo it’s sys-boot/syslinux.

Make sure the drive isn’t automounted.

First use fdisk to make a simple partition. You might be able to use parted or a fancy new way, but there can be trouble with that. This does present problems for very large drives. This should look something like this for an 8GB drive.

Device      Boot     Start   End          Blocks    Id  System
/dev/sdc1   *        2048    15826943     7912448   83  Linux

Update the kernel about any partition changes.

sudo partprobe -s /dev/sdd

Then make the file system. There is some chatter that journals (ext3 and ext4) do not help with USB drives. I tend to believe that. KISS.

sudo mkfs -t ext2 -L EXTUSB /dev/sdd1

Mount it.

sudo mount /dev/sdd1 /mnt/usb

It might be smart here to boot the installation disk along with the USB drive (or install Gentoo with chroot) and do the installation. When it is time to partition the drives, do it manually. Don’t format since you’ve already done that. Select the USB drive to mount as root. If there are any other drives on the system check to see that its swap is not automatically included in the fstab. No big deal if it is though.

Find an MBR. Check in /usr/share/syslinux/mbr*bin. If using syslinux from source, check ${PARENT}/syslinux-6.03/bios/mbr. Or if you love adventure, replace bios with efi64 or efi32. Put an MBR in place.

sudo dd if=/usr/lib/extlinux/mbr.bin of=/dev/sdd

Run extlinux. Yes, the man page confirms that it is designed to be pointed at a mounted directory’s mount point.

sudo extlinux --install /mnt/usb

Edit the /mnt/usb/extlinux.conf or copy another syslinux type loader’s config file into that file. That file needs to be there.

Hmm. It seems it’s harder than it should be to find an example of this. Here’s one.

/extlinux.conf
#TIMEOUT 30
#ONTIMEOUT mynumberoneos

DEFAULT mynumberoneos

#UI vesamenu.c32

LABEL mynumberoneos
    #MENU LABEL This is my favorite Linux.
    LINUX /boot/vmlinuz-3.16.0-4-amd64
    INITRD /boot/initrd.img-3.16.0-4-amd64
    APPEND root=/dev/sdb1
    #APPEND root=UUID=xxxxxx ro quiet

I’ve commented out lines that are there for reference if you’re trying to get a menu to work but are probably not needed for a simple boot loader.

Add a kernel and initrd as described in your extlinux.conf file. The kernels and paths were correct for the Debian I tested. This actually worked for me. With this I was able to boot an installed Debian system.

The Debian boot disk was not so successful. I was able to boot a Debian install CD’s kernel from an ext partition on a USB drive. The problem is that then the installer was confused about where to look for the install payload. Since there was no cdrom per se it had to look on the drives. Unfortunately that kernel seemed only capable of looking at FAT partitions so it couldn’t mount and find the full iso sitting right on the USB key. So close! This won’t work for installers, but unetbootin works fine for them, but it does offer hope of how to get USB drives to boot nicely with an arbitrary system installed.

Unetbootin

I really appreciate this tool because most bootable usb drives I’ve ever made were made with it. At the same time, I hate it’s mystical opaqueness. What is it really doing? How can I do those steps (perhaps in a customized way)? The big problem with Unetbootin is that it is designed for FAT file systems which are a bit of a joke for serious purposes.

Anyway, here’s the normal procedure for doing things like a normal person.

sudo apti unetbootin
sudo fdisk /dev/sdd  # Type is "c" W95 FAT32 (LBA) + bootable
sudo mkfs.vfat -n VFATUSB /dev/sdd1
sudo partprobe -s /dev/sdd
sudo mount /dev/sdd1 /mnt/usb/
sudo unetbootin

Testing The Speeds of Drives

Create a file of precalculated pesudorandomness ready to go from shared memory.

time dd if=/dev/urandom of=/run/shm/randomness bs=1M count=1000

Dump that on to the raw device. Obviously this will hose all contents so be careful. The fdatasync "conversion" makes sure that the data is properly on the device (not just promised in caches somewhere) before quitting with the final timing.

$ sync ; time sudo dd if=/run/shm/randomness of=/dev/sdd bs=1M count=1000 conv=fdatasync
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 197.921 s, 5.3 MB/s

real    3m17.936s
user    0m0.003s
sys 0m2.017s

Now that the data is there, see if it can be read back. Ideally the device should be unplugged and reinserted before this step to prevent Linux from knowing what it just wrote there and making some stupifyingly clever optimizations, which it often does.

$ sync; time ( sudo dd if=/dev/sdd bs=1M count=1000 | md5sum )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 48.4136 s, 21.7 MB/s
6871f9bd9893cb2b9ae97deaa6968444  -

real    0m48.426s
user    0m4.465s
sys 0m2.862s

Test what optimal MD5 calculating looks like. This can be subtracted from the read test since it’s the part of that test that’s not really doing any reading.

$ sync; time ( sudo dd if=/run/shm/randomness bs=1M count=1000 | md5sum )
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 3.43915 s, 305 MB/s
6871f9bd9893cb2b9ae97deaa6968444  -

real    0m3.453s
user    0m2.533s
sys 0m1.776s

So 48.426 minus 3.453 gives 44.973 seconds for 1GB or 22.2 MB/s pure read action.

What about along the whole drive? Is the writing speed even?

$ sync; for S in $(seq 0 250 7750); do echo $S; sync ; \
time sudo dd if=/run/shm/randomness of=/dev/sdd bs=1MB \
count=250 seek=$S conv=fdatasync; done

This writes 64kB at each place along the whole drive every 25MB.

sync ; for S in $(seq 0 400 124000); do echo $S | \
tee -a /run/shm/results; sync; \
dd if=/run/shm/randomness of=/dev/sdd bs=64kB \
count=1 seek=$S conv=fdatasync 2>>/run/shm/results; done

How to that’s how to test writing. To test reading is similar.

# sync ; for S in $(seq 0 5 160); do echo $S | \
tee -a /run/shm/results; sync; \
dd if=/dev/sdd bs=50MB count=1 skip=$S >/run/shm/blob\
2>>/run/shm/results; done

Another completely different test uses the mysterious hdparm.

sudo hdparm -Tt /dev/sdd

Dead?

Is the device dead? Here’s what that looks like.

$ sudo blockdev --report /dev/sdd
RO    RA   SSZ   BSZ   StartSec            Size   Device
ro   256   512  4096          0      8103395328   /dev/sdd

Note the RO flag is not "rw" as it should be. This device has freaked out and will only allow reads because it is trying to let you rescue your data but it can’t handle any more writing.

Read-Only Operation

Here’s a good paper on why one would want this and some ideas about how to go about getting it.

Some more practical information about this kind of architecture.

One inspiration is the Ubuntu guest accounts. This solves the problem of programs that need a writeable space (which, as with all writeable targets, must be in a tmpfs) but which already contains some things (which would clearly not already exist in tmpfs). This is a fundamental problem with using something like a window manager that expects a certain directory structure to be present (~/.config/mate/gtk or something like that) but must also be on a tmpfs.

Here’s a decent description of how Ubuntu guest accounts work.

Using RAM - tempfs

Here’s a good description of tempfs.

This kind of thing is possible.

# mount tmpfs /dev/shm -t tmpfs -o size=32m

Here’s an fstab of a system I was working on.

/dev/sda1 /               ext2 ro,errors=remount-ro 0
tmpfs     /tmpfstmp       tmpfs rw 0 0
tmpfs     /tmpfsvar       tmpfs rw 0 0
tmpfs     /tmpfsroot      tmpfs rw 0 0

The idea was to create these at boot, copy anything that needed to be there, and link to them so that they would all definitely disappear when the session ended. This is just some rough notes I’m preserving about this project. Don’t take it seriously yet; I might work on this some more.

.Xauthority

In normal situations, the ${USER}/.Xauthority file has some kind of information that lets users use the Xserver from diverse locations. If you’re using a read-only user directory however, this is annoying. The way to override it is to set an environment variable called XAUTHORITY to some writable location. Putting this in .bashrc seems to work.

This also apparently needs to be writable: /home/${USER}/.config/caja