bhyve UEFI shell, Linux, and grub
You've installed Linux (e.g. Debian or Ubuntu) into an UEFI bhyve guest on FreeBSD, only to be greeted with a message about PXE boot after restarting from the installer:
BdsDxe: failed to load Boot0001 "UEFI BHYVE SATA DISK BHYVE-48FF-992B-D5E0" from PciRoot(0x0)/Pci(0x4,0x0)/Sata(0x0,0xFFFF,0x0): Not Found
>>Start PXE over IPv4.
This article will describe what happened, and show you how to fix this and similar issues. I'll assume that you've started the virtual machine using the bhyve command and are familiar with its various options (see
man 8 bhyve).
In the message above, the UEFI firmware could not find an EFI bootloader, so it is attempting to perform a network boot. To skip this and simplify debugging, restart the bhyve virtual machine without a network adapter. You'll be greeted by the UEFI shell:
Our goal is to find the EFI bootloader file that the firmware could not locate.
Helpfully, the EFI shell lists all of the storage devices and partitions that it could find. There will usually be at least two partitions on the Linux virtual machine's hard drive: the first, a small partition holding EFI firmware, the second, the Linux system. In this case, the EFI partition is
FS0 (listed as "
HD1(1,GPT,…" i.e. the first GPT partition of the first drive) the main Linux partition is
BLK2 (listed as "
HD1(2,GPT,…" i.e. the second GPT partition of the first drive).
BLK0 is the drive itself (not considering the partition table) and
BLK3 here is the swap partition.
We can navigate into a filesystem by typing its name, followed by a colon. From there, familiar commands like
rm will be available. Let's see what was installed to the EFI partition:
Directory of: FS0:\
03/02/2021 16:54 <DIR> 4,096 EFI
0 File(s) 0 bytes
Directory of: FS0:\EFI\
03/02/2021 16:54 <DIR> 4,096 .
03/02/2021 16:54 <DIR> 0 ..
03/02/2021 16:54 <DIR> 4,096 debian
0 File(s) 0 bytes
Aha! The UEFI firmware expects to find the bootloader in
EFI/boot, not in
EFI/debian. Typically, bootloader paths would be set as UEFI variables, but
bhyve is unable to store these between reboots.
The easiest way to start the Linux guest from here is to execute the EFI bootloader inside the
Boot should now continue normally.
How can you avoid having to manually run the bootloader from the EFI shell? Here are three options:
grubunder the prefix that the UEFI firmware expects. From within the Linux guest, you can run
grub-install --bootloader-id=bootto install to
EFI/debian. You'll probably have to rename
bootx64.efias well (often, the EFI partition will be mounted by default at
- Don't use UEFI boot, rather, use
grub2-bhyveto boot using MBR / legacy boot.
- Implement support for UEFI variables yourself, and get this merged into FreeBSD.
- Use software like vm-bhyve to handle the details for you.