The Ubuntu installer stalls and won’t let you continue if you don’t have a physical optical drive installed - this is the case with many pizza-box rack mounted servers. You have to fake it out - here’s how:

First, if you have enough space on your installation USB drive, copy the Ubuntu iso onto there before you start. Yes, the iso file you just “burned” to the USB drive (You might not be using a USB drive to install, that’s ok, you can still use a USB drive to get the iso onto the server). If you don’t have enough space (for example if you’re using a 2G drive like I was), load up a second USB drive with the iso (format as FAT32 for best compatibility).

When you get to the screen “Load CD-ROM driver from removable media?”, switch to another console (Alt-F2) to mount the iso as a CD drive. First mount the USB drive, then mount the iso from the drive onto a separate mount point.

mkdir /mnt/usb /mnt/iso
# use ls /dev/sd* to see your mounted partitions.
mount –t vfat /dev/<usb drive, often sdb1> /mnt/usb
mount –t iso9660 –o loop /mnt/usb/ubuntu.iso /mnt/iso

Switch back to the installer (Alt-F1), choose to manually select device, then type /dev/loop0. The installation should continue flawlessly.

Thanks goes to this thread for pointing me in the right direction earlier today.