English Forums > Tutorials and FAQs

OPN as a PXE boot server

(1/1)

cookiemonster:
This mini tutorial is intended to setup a PXE boot server in OPNSense.
The clever parts I have used with permission are documented by Kraileth in his blog:
Links:
https://eerielinux.wordpress.com/2021/01/24/multi-os-pxe-booting-from-freebsd-12-introduction-pt-1/
https://eerielinux.wordpress.com/2021/01/30/multi-os-pxe-booting-from-freebsd-12-required-services-pt-2/
https://eerielinux.wordpress.com/2021/02/05/multi-os-pxe-booting-from-freebsd-12-pxe-menu-and-bsd-pt-3/
https://eerielinux.wordpress.com/2021/02/20/multi-os-pxe-booting-from-freebsd-12-linux-illumos-and-more-pt-4-2/
All I’ve done is adapted it to OPNSense and I made a couple of choices for my needs and infrastructure availability.

What we’ll have at the end of this tutorial is OPNSense acting as a PXE boot server that can be used for clients to network boot on a LAN interface. As an working example we’ll be serving a FreeBSD 12 boot option. I am using OPNsense 21.7.2_1-amd64. An important note is this is for BIOS boot, not UEFI.

The only additional dependency required is a plugin “os-tftp” found in the “default” mirror of OPNSense i.e. OPNSense package repository.

As a baseline I describe my setup. I don’t have VLANs and my flat network is in the 192.168.5.0/24 range. OPN has a lan IP of 192.168.5.1. I have a FreeBSD-based NAS with a reserved IP of 192.168.5.5 that has an available webserver running on port 8081. I will use this storage+internal webserver to serve the large isos. It is perfectly possible and simpler to host them in OPN but I have limited storage on my firewall appliance and I don’t want to run a webserver on it either.
I use DHCPv4 and not v6.
I use Unbound for name resolution.

Prerequisites:
A shell account that can elevate permissions with sudo on OPN.
A shell account that can elevate permissions with sudo on a local webserver.
Internet connection to download some necessary files. I use a separate computer and then transfer to the firewall, to avoid costly mistakes. Some will prefer to do it all on the firewall itself.
Some familiarity with basic shell commands and utilities like wget, nano, etc.

Now we start.
1. Install the tftp plugin in OPNSense
Go to System > Firmware > Plugins. Search for os-tftp. Click on the + sign at the end of the row to install it.
You will get a message “The root folder for transfering files is /usr/local/tftp.”
Once the WebUI refreshes you’ll have an entry in Services > TFTP > General where the service can be enabled or disabled.

On a shell connect to OPNSense and verify the path exists. For me it didn’t so I created it

--- Code: ---$ sudo mkdir -p /usr/local/tftp
--- End code ---

Then back on the WebUI I enabled the tftp service and entered my OPN LAN address 192.168.5.1 and save. Now we have a tftp server listening. To check:

--- Code: ---$ sudo sockstat -4l
Password:
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS     
root          in.tftpd            48436 4       udp4   192.168.5.1:69                    *:*

--- End code ---
I did root around the system to figure out how the plugin sets things up but I couldn’t find much. I was fumbling in the dark with this. I see it does not use inetd but I can’t yet find it’s config. I found an entry in etc/rc.d/ftpd but not much else. Nevertheless it is running and working fine for our purposes, and starts after reboot.
If someone finds where the control and config files are, let me know.

1.1 Create the pxe boot files being given by ftpd


--- Code: ---sudo mkdir -p /usr/local/tftp/pxelinux.cfg
sudo nano /usr/local/tftp/pxelinux.cfg/default
--- End code ---

Insert the following and save:

DEFAULT vesamenu.c32
PROMPT 0
MENU TITLE PXE Boot Menu (Main)

LABEL bsd-oses
   MENU LABEL BSD Operating Systems
   KERNEL vesamenu.c32
   APPEND pxelinux.cfg/bsd


Now for a test, from a linux or FreeBSD machine on the LAN do:

--- Code: ---$ cd /tmp
$ tftp 192.168.5.1
tftp> get pxelinux.cfg/default
Received 190 bytes in 0.1 seconds
tftp> quit
$ rm default
--- End code ---

That confirms our tftp daemon on OPN is serving the files correctly. I was expecting to need to open the port 69 but it seems the default pass rule from the LAN takes care of it.
You will need to adjust the firewall rules if you don’t have that permissive rule in place.

Now we need to create the referenced file “bsd”:

--- Code: ---$ sudo nano /usr/local/tftp/pxelinux.cfg/bsd
--- End code ---
insert this:

MENU TITLE PXE Boot Menu (BSD)

LABEL main-menu
   MENU LABEL Main Menu
   KERNEL vesamenu.c32
   APPEND pxelinux.cfg/default

This step 1.1 has created a default file and a bsd file, both inside a subdirectory under our tftp root.
Now we’re ready to get those files they reference.

2. Get the PXElinux files.
Ordinarily I do this on a separate machine and then transfer the files but on this ocassion I’ll do it directly on OPN.


--- Code: ---$ cd /tmp
$ sudo pkg fetch -y syslinux
$ sudo mkdir -p /tmp/syslinux
$ sudo tar -C /tmp/syslinux -xvf /var/cache/pkg/syslinux-6.03.pkg
--- End code ---

Now we copy the required files:


--- Code: ---$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/core/lpxelinux.0 /usr/local/tftp/pxelinux.0
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/com32/elflink/ldlinux/ldlinux.c32 /usr/local/tftp/
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/com32/menu/vesamenu.c32 /usr/local/tftp/
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/com32/lib/libcom32.c32 /usr/local/tftp/
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/com32/libutil/libutil.c32 /usr/local/tftp/
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/com32/modules/pxechn.c32 /usr/local/tftp/
$ sudo cp /tmp/syslinux/usr/local/share/syslinux/bios/memdisk/memdisk /usr/local/tftp/
$ sudo rm -r /tmp/syslinux
--- End code ---

At this point we have on our tftp server the files required for menus and the files required to get the clients to boot to a prompt for an installation. However we don’t have an installation media to offer yet.
For our example we are now going to get a FreeBSD 12.2 distribution iso.
Kraileth makes a great explanation of why for FreeBSD we need an mfsBSD https://mfsbsd.vx.sk/ image that can network boot.

3. Getting the image media available on our webserver.
TFTP is not meant to be used to transfer large files like distro isos. It is too slow, transferring files by limited block sizes, and the clients need to acknowledge each block, taking a long time and prone to timeouts. Fortunately we can use a local webserver. I have a XigmaNAS server that simply requires me to tick a box to enable the webserver service, choose a port and a root directory.



I chose to use http on port 8081 and the NAS has an ip of 192.168.5.5 so that’s the IP the webserver is bound to, on port 8081. We’ll need this information in a little bit.
The webroot is “/mnt/Deimos/www”  .
3.1 Now I need to connect to a shell on the NAS:

--- Code: ---$ ssh <myuser>@nasip
--- End code ---

Then create a subdirectory under the root:

--- Code: ---$  sudo mkdir -p /mnt/Deimos/www/pxe
--- End code ---

I think I had to change ownership before proceeding with subdirectories. This part is from memory, it might be needed as a later step, i.e. after the whole path was created.

--- Code: ---$ sudo /usr/sbin/chown -R www:www /mnt/Deimos/www/pxe
--- End code ---

Then finally creating them with a sane structure:

--- Code: ---$ sudo mkdir -p /mnt/Deimos/www/pxe/bsd/fbsd/amd64/12.2-RELEASE/
$ sudo fetch https://mfsbsd.vx.sk/files/iso/12/amd64/mfsbsd-12.2-RELEASE-amd64.iso -o /mnt/Deimos/www/pxe/bsd/fbsd/amd64/12.2-RELEASE/mfsbsd.iso
$ sudo gzip -9 /usr/local/www/pxe/bsd/fbsd/amd64/12.2-RELEASE/mfsbsd.iso
--- End code ---

3.2 Now we need to edit the pxelinux.cfg/bsd file and append:

LABEL fbsd-pxe-install
   MENU LABEL Install FreeBSD 12.2 (PXE)
   MENU DEFAULT
   KERNEL memdisk
   INITRD http://192.168.5.5:8081/pxe/bsd/fbsd/amd64/12.2-RELEASE/mfsbsd.iso
   APPEND iso raw

4. Final changes in OPNSense.
This was trial and error & some additional reading online. I can’t explain how or why too well but it works for me.
I don’t use Ipv6, only v4.
I go to Services > DHCPv4 > LAN
4.1 In “TFTP server” field I entered the LAN ip of OPN: 192.168.5.1
In the “Set Bootfile” field I entered: pxelinux.0


To troubleshoot permutations of these fields I used the nmap macro that @Kraileth suggests on one is his posts to check DHCP responses. I used to get a bootfile and server offered:
From a linux machine on my lan:


--- Code: ---$ sudo nmap --script broadcast-dhcp-discover
[sudo] password for cookiemonster:

Starting Nmap 7.60 ( https://nmap.org ) at 2021-10-01 23:13 BST
Pre-scan script results:
| broadcast-dhcp-discover:
|   Response 1 of 1:
|     IP Offered: 192.168.5.189
|     DHCP Message Type: DHCPOFFER
|     Server Identifier: 192.168.5.1
|     IP Address Lease Time: 5m00s
|     Subnet Mask: 255.255.255.0
|     Router: 192.168.5.1
|     Domain Name Server: 192.168.5.1
|     Domain Name: moomooland
|     Bootfile Name: pxelinux.0
|_    TFTP Server Name: 192.168.5.1
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 1.69 seconds
--- End code ---

5. Testing with an actual client.
For this I used VirtualBox. I created a new Virtual machine.
Type: BSD
Version: FreeBSD (64 bit)
Memory: 1 GB; 18 GB vdi storage
Boot Order: Network, Hard Disk, Optical
Network: Bridged Adapter , Name: the name of the adapter on the host that is plugged into the LAN    that will connect to OPN.
With these settings I am saying, try to boot from the network and that network is the one where OPN is the DHCP server, giving the ip address and the network boot options.



Then I can start the Virtual Machine and verify:





At some point I had to do packet captures to verify the client to figure out that there was a tftp transfer despite the client complaining that thee was no file received.
The problem was that I had entered the “default” or “bsd” file. I didn’t realise it had to be the pxelinux.0 one that was needed in OPN.

Navigation

[0] Message Index

Go to full version