Messages

Thanks for the reply Patrick. I think I have my answer and that root then drives the rest of the codebase.

Maybe my hardware choices are the issue. VM performance is not great compared to Linux, driver issues abound.

Dedicated HW like the link you provided I can understand.

I just wonder how many of the userbase fit into the category of enterprise users. I suspect a large proportion of the community are home network or similar to myself users.

I run it on the edge of my SMB lab and home Lab. Currently, it is sitting on a Dell Rackmount ESXI host. If we had better internet it might be an issue but we are not blessed with that in the rural locations.

For example across the internal ESXi vswitch I get a max of .63 Gb and across the lan on 1G infrastructure .55 Gb. We spent a long time following all the tweaking guides to get the best lan performance we can. As I said on the WAN side we don't really care as our backhaul is sub 100Mb.

Linux Vm's across the switch are running around 9Gb and across the lan at Gb wire speed.

It's just an example I am sure there are many more.

Thanks for taking the time to respond.

Cheers
Spart

What makes linux so special???
if you are looking for small amount of foot print, then you go with NetBSD. If you want security, then your choice is OpenBSD. For enterprise class POWERHOUSE, you go with FreeBSD.

BSD just takes a lickin' and keeps on ticking...

Now if you want GUI... then you can go back to untangle and work on making it look like the way you like it... :D

Obviously a xBSD fan. But your assertions may have been valid 10 years ago around security and enterprise but not today and just stating them as fact is misleading. I like OpnSense but it is built on sinking foundations.

There are other foundations that are community supported like debian for instance and more capable and performant with massive global footprint and support ecosystems. If it's really pf that's the anchor then there is no way forward. 

I was really just trying to understand if there was a deal breaker dependancy and maybe it is pf. I have no understanding of how the product architecture is structured and whether for instance the UI is closely or loosely coupled or integrated. If you take OpenWRT you can hack it from the command line or install luci from the standard package management and config via web UI. You can do similar with OpnSense core.

And why does anyone who asks these type of questions just get told to * off back to where they came from :)

Cheers
Spart

Cheers
Spart

OPNsense's basic architecture is built on the pf packet filter - which is BSD only.
Of course you can build a Linux based firewall, but it wouldn't be OPNsense.

OpenWRT and IPfire exist.

They do exist but are clunky. I came here from Untangle.

These days netfilter is built into the kernel and nftables is the new standard for a lot of the enterprise linux OS vendors. massive dev effort across that community.

Are you saying that the main reason for sticking with BSD is pf?

Cheers
Spart

I am sure this topic has come up before but, I was wondering why the dependency on BSD.

These days a base Ubuntu server which is capable of routing is using <200 MB of ram and runs on just about any type of hardware with decent NIC drivers that have very active development.

It would be awesome to have all of the OpnSense goodness on top of an enterprise grade mainstream Linux server OS that has mucho dinero spent on development.

Maybe it's just history and legacy but I think it's holding OpnSense back!

Cheers
Spart

What about RockPi-X or Odroid? Has anyone had success with one of these?

RockPi-X suffers from the same limitation as an ARM based Pi: only one Ethernet port. So you have to use USB-Ethernet, which I would rather avoid. Or bring an extra switch and trunk. But then you could just use an EdgeRouter-X or similar ;)

Interesting comment regarding not using USB based ethernet. The testing I did on freebsd on Rpi4 showed the USB3 ethernet adaptor was just as good as the ethernet port which as we all know is not wire speed! But unless you have GB Wan connections more than good enough at about 700mb. If you have GB WAN etc. then you would be using more capable HW anyway.

Admittedly the xBSD ethernet drivers are S**t in general compared to the linux drivers. Running a basic Ubuntu 22.04 server on the rpi4 I could run both the internal GB Nic and the USB3 NIC at wire speed.

No point in escalating to the BSD devs as no one even picks up the reports and acknowledges the drivers have issues so they never make it into the dev pipeline.

As OpnSense is running on ESXI I had to create a dedicated Port Group and attach the OpnSense LAN adaptor to it and reject promiscuous mode on that PG.

Activating NtopNG still sets promiscuous mode inside the guest but it now only sees traffic on its PG and not on the vswitch.

I hope this helps others that may see this kind of issue.

Cheers
Spart

If I turn off ntopng the problem goes away. I believe it is related to promiscuous mode being set when ntopng is enabled.

Any advice appreciated.

Cheers
Spart

No just massive amount of log spam that is halting my opnsense router when it runs out of space ~12G of logs in 6 hours.

It seems to be routing lan traffic that is lan to lan. I have no idea why it is even going through the opnsense router. It should simply be lan to lan traffic.

I thought it might be due to promiscuous mode. But that is off on the Lan.

opnsense is running on esxi 6.7

Any help is appreciated as I am having to reset logs multiple times a day.

Cheers
Spart

Hello,

I updated to the latest version today. I noticed I am seeing traffic stats for lan to lan traffic. I thought at first something was sending data out from the lan. But then realised it was showing stats for direct lan to lan connections!

Very confused by this as this particular server is a Video Surveillance server with cameras talking to it directly across the lan.

My understanding of the traffic graph is it is showing traffic traversing the opnsense interfaces.

How am I seeing traffic that is going from an IP camera to the server across the lan?

Cheers
Spart

Talking to myself probably.

I managed to get the 22.1.10 image to boot on my RPI4 after connecting a ttl adaptor and seeing what was going on. Weirdly I think it was a corrupt write operation to the card.

I decompressed the image first then wrote it to the sd card rather than let Ubuntu disks app do it on the fly.

That worked and I had a booting image.

It is very snappy compared to my vmware OPNSense with much more resources.

Is it possible to get a build of the latest 22.7 as it has a few inclusions we need one being APCUPSD service.

But this has been a great surprise and I am a little shocked at all the negative comments regarding running OPNSense on the RPI4 I am using the native GbE adaptor as the lan adaptor and a J5Create USB3 Adaptor as the wan adaptor.

It uses the axe driver and is recognised and configed perfectly and seems to work without any issues.

As this seems to be a 'standard' aarch64 build it would be great to see this as a standard build from OPNSense ARM devices are proliferate and incredibly performant for the power footprint.

Thank you to yrzr for all the work producing these builds and the corresponding installation and configuration notes.

This is the link I followed https://www.yrzr.tk/opnsense-22-for-aarch64/ to get a working build.

Cheers
Spart

Can anyone point me at a working RPI4 22.x build please.

Just to add my RPI4 boots first time with the standard freebsd 13.1 aarch64 image.

Cheers

Sorry for the late reply!
Was busy with other stuff lately, had little to no time and to totally forgot to reply.

Hello @efetropy,

Can you share the patches required to build opnsense 22.1 for a Raspberry Pi 4? I'd like to help with this.

I'm just getting started and have finally understood how to build it but missing the specific configuration for RPI4.

Dear all,

Currently, I have a CM4 (with 4GB RAM and 8GB eMMC and no Wifi) and the DFRobot IoT Router Carrier Board (https://www.dfrobot.com/product-2242.html).

I am interested to compile an OPNsense build for this hardware but I am a little bit lost about it's potential compatibility.

Can someone confirm me that's this will be good or not?

Many thanks in advance!

There is already an image built by @yrzr for the RPi which can be found here:
https://ftp.yrzr.tk/opnsense/FreeBSD%3A13%3Aaarch64/22.1/images/

A generic aarch64 vmdk image is also available (for e.g. for proxmox, esxi fling or whatever you're using).

The RPi 4B should work without problems (if not, just get some different boot files).

CM4 is a different case. It does work with the official I/O board albeit PCIe issues (see bugzilla).
Your carrier board states ETH2: PCI Express 1000BASE-T NIC (based on RTL8111). RTL8111 drivers are in fact available, but
it will be a hit or miss and you might not be able to the use the 2nd ethernet port due to the mentioned PCIe issues.
Furthermore, you have to figure out how to deal with the dtb file. Worst case scenario, you might have to make it compatible with freebsd.

Trying to get the 22.1.10 build working on my RPI4B 4GB but it does not book.

Can you advise how to get a working build for the RPI4?

Cheers

I have a decommissioned RPI4 that I am wanting to run OPNSense on. I have tried this build for the RPI4 but it does not boot.Blink code suggests it can't find the kernel.img.

https://www.yrzr.tk/opnsense-22-for-aarch64/

Can anyone suggest how I get a build up and running on the RPI4? At this location, I only have a 40/10 VDSL service so the USB3 J5 Create nic should be fine.

Really would like to play with a working build. I saw earlier in the thread someone said they had a 22.7 build running.

I have tested my hardware with the standard FBSD aarch64 image and it boots first time so I know the hardware is good.

Any help is appreciated.

Cheers

Did you actually try the VMXNET3 driver? Unfortunately I only have an ADSL connection here in the UK but when I lived in France I had a full Gigabit fibre connection and I ran the VMXNET3 drivers on OPNsense for about 8 years and never had a slow download  and always the full speed that was also on ESXi 6.7 and also updated to ESXi 7.

Yes, it is running with that adaptor now but performance is not optimal. The issue it seems has nothing to do with VMWare it is the FBSD driver. There are many open 'bugs' and no action from the dev team.

We are limited in our options on this one. There is no hardware to spin up at the site. The e1000 is worse than the VMXNET3.

We may have an option of installing a dual port NIC apart from the standard quad port netXtreme already in it. We could in theory pass it through esxi to the OPNSense VM and use that.

Looking through the HCL for FreeBSD 13.1 (https://www.freebsd.org/releases/13.1R/hardware/) the vmx driver is not even listed. SO maybe they dropped support for it quietly!


Cheers

Talking to myself I know but after a full day of testing and reading endless posts/kb's etc. I am  not really any nearer to a solution.

This is the latest release of OPNSense installed clean on a new esxi 6.7 VM with 4VCPU and 8GB memory. Host is a dell R720 with dual 2650 v2 and a 4 port Broadcom BCM5720.

The test machine is a I7 8700K 12 CPU's and 32Gb ram.

All linux VM's run at full 1GB wire speed across the lan with iperf3 testing.

The OPNSense VM varies in speed but is between 250 - 350 Mb/s slower. Best speeds seems to be with LRO on and tunable hw.pci.honor_msi_blacklist = 0

With LRO off and the tunable removed then speed is roughly a 1/3 of the linux machines some of which are running old versions like 16.04 for instance.

This system is in the sticks not in civilisation so every Mb/s counts when we are doing remote backups etc.

Cheers
Spart