1
Tutorials and FAQs / Making OPNsense more useful with custom-built packages
« on: February 25, 2021, 06:05:28 am »
Part 1: Introduction
Situation
FreeBSD offers over 30,000 ready-to-install binary packages. OPNsense provides only a couple hundred (just over 700 right now). This is because the FreeBSD project maintains a general purpose operating system that needs a wide range of software available to be most useful. OPNsense as a much smaller effort concentrates an staying light and focusing on what's common to do on the special-purpose system that it is.
For people who'd like to do a little more with their OPNsense, it suggests itself to extend it with additional packages. My router running it is an always-on device after all and perfectly fit to host my Git repositories for home use for example. Having a candidate that could provide services always available on my net while the system is running on embedded hardware to achieve low power consumption is very intriguing.
When I decided to do this project next after finishing the previous one (in early January), I did not know of the Community Repo (https://forum.opnsense.org/index.php?topic=20827.msg97031#msg97031). So this was not done deliberately as an action to harm that effort. In fact it is much easier to use pre-built packages than going down the route that I describe here. Still some people might be interested in doing it for themselves or have organizational rules that make trusting third party packages complicated (or impossible).
Why not use foreign or offer native packages?
So why not just install a couple of FreeBSD packages on the system? OPNsense is FreeBSD-based after all, right? Yes and no. Today OPNsense is an indirect FreeBSD-derivative system. The upstream for the OS is HardenedBSD, a security-focused close fork of FreeBSD that regularly pulls the latest changes from its mother.
But OPNsense is not just HardenedBSD with a couple of additional packages on top. There are some pretty invasive changes from system startup to the running OS. It's meant to be administrated entirely (or mostly) via its GUI whereas vanilla FreeBSD and HaredenedBSD are configured in classic Unix style on the command line and via editing text files.
Installing FreeBSD's packages on OPNsense is a particularly bad idea. But even using the HardenedBSD packages is not a good one. It's closer to OPNsense but incompatibilities probably do exist and in general native packages are the most sensible solution anyway.
Then why doesn't the OPNsense project simply ship most of or all the FreeBSD packages? Most of them could probably be built for OPNsense, too, after all. Well, maintaining FreeBSD's huge package repository is a daunting task. And even if most of the work is already done by FreeBSD and HardenedBSD, building all those packages is very demanding in resources. The FreeBSD project maintains build clusters of pretty beefy machines building packages all the time. Building an entire set of packages takes those servers a couple of days (when done with that, they begin the next build) for each architecture.
Even if the OPNsense project had the resources to do this, it would largely be wasted compute power. Who would ever run Chromium or LibreOffice on his or her firewall boxes?? Those packages (and their dependencies) alone take hours to build even on modern hardware. Considering to do this is not feasible at all. Therefore OPNsense dedicates its resources on the commonly used packages and is willing to add more if people request it (and the package is not too heavy).
OPNsense does offer easy access to FreeBSD's ports collection however. This "PFC" is a tree of directories which contains various ports. If you don't know what that is: Think "build recipes" for all the software available as packages (and more that e.g. may not be distributed in binary for license reasons). Technically these are Makefile fragments and additional files (patches, init scripts, ...). These ports can be used to build, package and install software directly on OPNsense.
There's a couple of issues with this, though. While it works perfectly well, this traditional way of installing software on FreeBSD is not ... clean. It needs to make available all of the build-time dependencies for your software, too, after all. Sure, you could manually clean up after it, but that's not much fun. Building software like this also means that what's already on the system can influence the result - which is not pretty. It also doesn't scale well: If you are responsible for more than a couple of OPNsense installations, building packages on all the devices (which are often not extremely fast) is time-consuming and wasting resources.
Solution: Building your own additional packages - properly
While you can still build all your software directly from ports on FreeBSD, the project has solved the above issues by providing a better way. There are two programs that cleanly build packages by creating pristine build environments for each package and tearing it down afterwards.
The newer tool, Synth, is more convenient to use but is in maintanence mode and is written in a less comon language (Ada) that needs a special compiler to build. I like it, but then I'm part of the successor project called Ravenports and thus not cannot claim to be neutral on this matter. The other is Poudriere. It's the officially supported build tool for FreeBSD and thus probably the better fit here despite being more complicated to work with.
If you are interested in making additional programs available on your OPNsense box, you must - at least for now! - be somewhat confident in your basic command line skills. What I'm describing here is not possible using the GUI - unless the community at some point decides to offer a proper plugin.
But enough pie in the sky already: Let's get our hands dirty! I'll try to be gentle and explain things along the way, but if you have no shell experience at all (I haven't forgotten you, Windows users. No guarantees though that I'll succeed in explaining everything sufficiently, but I'll try) this is probably not the kind of journey you'll enjoy. Better do at least a little preparation work beforehand.
Situation
FreeBSD offers over 30,000 ready-to-install binary packages. OPNsense provides only a couple hundred (just over 700 right now). This is because the FreeBSD project maintains a general purpose operating system that needs a wide range of software available to be most useful. OPNsense as a much smaller effort concentrates an staying light and focusing on what's common to do on the special-purpose system that it is.
For people who'd like to do a little more with their OPNsense, it suggests itself to extend it with additional packages. My router running it is an always-on device after all and perfectly fit to host my Git repositories for home use for example. Having a candidate that could provide services always available on my net while the system is running on embedded hardware to achieve low power consumption is very intriguing.
When I decided to do this project next after finishing the previous one (in early January), I did not know of the Community Repo (https://forum.opnsense.org/index.php?topic=20827.msg97031#msg97031). So this was not done deliberately as an action to harm that effort. In fact it is much easier to use pre-built packages than going down the route that I describe here. Still some people might be interested in doing it for themselves or have organizational rules that make trusting third party packages complicated (or impossible).
Why not use foreign or offer native packages?
So why not just install a couple of FreeBSD packages on the system? OPNsense is FreeBSD-based after all, right? Yes and no. Today OPNsense is an indirect FreeBSD-derivative system. The upstream for the OS is HardenedBSD, a security-focused close fork of FreeBSD that regularly pulls the latest changes from its mother.
But OPNsense is not just HardenedBSD with a couple of additional packages on top. There are some pretty invasive changes from system startup to the running OS. It's meant to be administrated entirely (or mostly) via its GUI whereas vanilla FreeBSD and HaredenedBSD are configured in classic Unix style on the command line and via editing text files.
Installing FreeBSD's packages on OPNsense is a particularly bad idea. But even using the HardenedBSD packages is not a good one. It's closer to OPNsense but incompatibilities probably do exist and in general native packages are the most sensible solution anyway.
Then why doesn't the OPNsense project simply ship most of or all the FreeBSD packages? Most of them could probably be built for OPNsense, too, after all. Well, maintaining FreeBSD's huge package repository is a daunting task. And even if most of the work is already done by FreeBSD and HardenedBSD, building all those packages is very demanding in resources. The FreeBSD project maintains build clusters of pretty beefy machines building packages all the time. Building an entire set of packages takes those servers a couple of days (when done with that, they begin the next build) for each architecture.
Even if the OPNsense project had the resources to do this, it would largely be wasted compute power. Who would ever run Chromium or LibreOffice on his or her firewall boxes?? Those packages (and their dependencies) alone take hours to build even on modern hardware. Considering to do this is not feasible at all. Therefore OPNsense dedicates its resources on the commonly used packages and is willing to add more if people request it (and the package is not too heavy).
OPNsense does offer easy access to FreeBSD's ports collection however. This "PFC" is a tree of directories which contains various ports. If you don't know what that is: Think "build recipes" for all the software available as packages (and more that e.g. may not be distributed in binary for license reasons). Technically these are Makefile fragments and additional files (patches, init scripts, ...). These ports can be used to build, package and install software directly on OPNsense.
There's a couple of issues with this, though. While it works perfectly well, this traditional way of installing software on FreeBSD is not ... clean. It needs to make available all of the build-time dependencies for your software, too, after all. Sure, you could manually clean up after it, but that's not much fun. Building software like this also means that what's already on the system can influence the result - which is not pretty. It also doesn't scale well: If you are responsible for more than a couple of OPNsense installations, building packages on all the devices (which are often not extremely fast) is time-consuming and wasting resources.
Solution: Building your own additional packages - properly
While you can still build all your software directly from ports on FreeBSD, the project has solved the above issues by providing a better way. There are two programs that cleanly build packages by creating pristine build environments for each package and tearing it down afterwards.
The newer tool, Synth, is more convenient to use but is in maintanence mode and is written in a less comon language (Ada) that needs a special compiler to build. I like it, but then I'm part of the successor project called Ravenports and thus not cannot claim to be neutral on this matter. The other is Poudriere. It's the officially supported build tool for FreeBSD and thus probably the better fit here despite being more complicated to work with.
If you are interested in making additional programs available on your OPNsense box, you must - at least for now! - be somewhat confident in your basic command line skills. What I'm describing here is not possible using the GUI - unless the community at some point decides to offer a proper plugin.
But enough pie in the sky already: Let's get our hands dirty! I'll try to be gentle and explain things along the way, but if you have no shell experience at all (I haven't forgotten you, Windows users. No guarantees though that I'll succeed in explaining everything sufficiently, but I'll try) this is probably not the kind of journey you'll enjoy. Better do at least a little preparation work beforehand.