Docker in the OPNsense environment?

[Paul.C]

Does anyone have experience running docker under FreeBSD/OPNsense?

I'm not expecting docker & OPNsense to integrate, I'm just looking for advice about any conflicts or gotchas that others have run into.

I'm thinking that might be a better way to go for installing Ubiquiti Controller, rather than a one-off manual install.

Thanks,

- Paul

[Antaris]

I also look in this direction, because it's more easy and versatile for developers to maintain just Docker rather than many plugins for third party services like Ubiquiti Controller.
Here is an experimental implementation:

https://wiki.freebsd.org/Docker

[AveryFreeman]

1) Docker runs on Linux.  FreeBSD is not Linux - you might be able to get Linux container infrastructure to work via compatibility layer, but it's likely to be wonky

2) Jails are FreeBSD OS containers similar to LXC/LXD and are over a decade old and well supported.  If you're looking for an existing resource base you can explore iX System's take on 'containers', it may be rather trivial to adapt to OPNSense:

https://www.ixsystems.com/documentation/freenas/11.2/plugins.html

https://download.freenas.org/plugins/9/x64/

3) If you want the full web GUI for installing these containers you could try installing OPNSense on FreeNAS - it would be totally unsupported and get trashed if you upgrade the FreeNAS OS, but theoretically it's possible:

https://github.com/opnsense/update#opnsense-bootstrap

4) IMO I think this person had the right idea when they said run a VM environment w/ an OPNSense VM and then another VM for your container infrastructure (e.g. Ubuntu 18.04 VM or something) - I'm using ESXi.  You could just run Linux and virtualize OPNSense in a KVM VM, too:

https://forums.freenas.org/index.php?threads/freenas-bhyve-and-opnsense.56511/

https://wiki.opnsense.org/manual/virtuals.html

[Antaris]

I also use OPNsense not only baremetal, but on ESXi and Proxmox in different scenarios.

Dockers are very popular now and if there was an option to add some dockers to baremetal OPNsense will be more versatile.

[AveryFreeman]

 

[chemlud]

@Avery: Explain it to the cowboy in a way he will understand.

@Antaris: No way.

[Antaris]

@both: Strange reincarnation...
@Avery: If there is virtualisation option in OPNsense in some scenarios will be good to spare one OS layer(PVE or ESXi etc...) it it's a lighweight case.
@chemlud: Explane what is not clear and i will try to make you understand

[marshalleq]

I haven't looked into why exactly docker doesn't run on BSD, with the code being open, I really do wonder why not.  I see even FreeNAS has recognised this and is well through a linux kernel implementation that you can swap out from your BSD kernel somehow.  They are recognising the groundswell around docker and getting pushed into it by the market I expect.

All I want is to use Opnsense for a load balancer for about 14 services including a couple of web servers, and have lets encrypt handled by that too.  I have a docker that does it now and expect it to be painful to do in Opnsense, mainly because of letsencrypt.  I've actually started looking at other firewalls because of this one problem, which I really don't want to do because I'm very happy with opnsense.

[banym]

@marshalleq as you have mentioned correctly. The code is open. Go for it and port it yourself or hire someone to do so.

BTW. Docker is only one implementation of a container. The popularity was high on Linux but the business use cases are moving away from docker itself, to Kubernetes, OpenShift or similar solutions. Even VMware ESXi an now host containers. SystemD btw. can utilize the container building blocks itself without docker. RedHat does not push Docker but Podman, which btw. has a better design and is Docker file compatible.

On FreeBSD direct we have Jails for many years now. Not the best user interface but a good solution for FreeBSD.

This is a firewall project and the focus is to deliver a stable and good firewall.
If you need some more plugins to add some functionality go for it. But adding virtualization to a firewall is not a good option.
Most people here in the community are using OPNsense for what is was designed to do and the project and software works so good because it does not change its focus.
Feel free to use the framework for GUI and system control for a FreeBSD based virtualization appliance, I think this would be possible within 3-6 month you could have a working PoC with some nice features. In this project the focus is networking and firewalling. But even if you have such a solution. Real security and virtualization does NOT go together. Virtualization is not insecure but it is far away to be considered secure. There are to much layers involved that are constantly changed and modified to meet up to date requirements, support modern hardware, etc.

In my opinion not a good idea.

[dia4]

Hi,
if it helps, I'm currently run OPNSense in a VM on a FreeNAS system (FreeBSD). I agree that it is not the best solution about security but for my home environment i think is enough at the moment.
For the company where i'm working i bought a Deciso OPNsense A10 Dual Core Entry Level firewall.

Ciao

[jodumont]

Hey guys and girls, we are at the era of microservices
running everything in a box/os is not the way
running everything under one box [windows/appliance/linux/bsd] is the old model; so 1990.
Segmenting the usage at least per VM or container is more 2010, by services/pod it is more 2020.

So if you run OPNsense under FreeNAS, Proxmox, ESXi or Hyper-V you started well
now you should make another VM for your docker or better make a kubernetes cluster (try k3os to start it is easy).
then use OPNsense as gateway/firewall and add a proxy service (haproxy or nginx) on it to redirect the traffic on your docker machine/kubernetes cluster.
this is the way to do it.

now to make your kubernetes cluster resilient you will need a NAS: look at FreeNAS or OpenMediaVault for that and share a directory via iSCSI or NFS, or even SMB to start.

[bartjsmit]

we are at the era of microservices

Fine for applications but k8s namespace separation is not strong enough for security appliances like OPNsense. Setting up a separate cluster for your firewall is likely overkill.

Virtual machines are the sweet spot for me - easy enough to increase resources quickly and very well isolated.

Bart...

[Nnyan]

I was running my fw/gateway virtualized on ESXi on one big box for some years.  But having everyone in one box was a bit limiting so I then I went with multiple supermicro boxes for virtualization.  In the end I found some things just ran better bare metal (firewalls, unraid, TrueNAS, etc...) so I stopped worrying about trying to cram everything in a VM/container/microservice and just deployed it to what worked best for me.